2020-04-10T06:49:20.651 INFO:root:teuthology version: 1.0.0-0f93372 2020-04-10T06:49:20.652 DEBUG:teuthology.run:Teuthology command: teuthology /tmp/teuthology-worker.Zn341y.tmp -- --verbose --lock --description rbd/mirror/{base/install.yaml clients/{mirror-extra.yaml mirror.yaml} cluster/{2-node.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-low-osd-mem-target.yaml supported-random-distro$/{ubuntu_latest.yaml} workloads/rbd-mirror-snapshot-workunit-journaling.yaml} --name teuthology-2020-04-09_06:00:03-rbd-octopus-distro-basic-smithi --block --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2020-04-09_06:00:03-rbd-octopus-distro-basic-smithi/4937628 2020-04-10T06:49:20.718 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-04-10T06:49:20.792 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2020-04-09_06:00:03-rbd-octopus-distro-basic-smithi/4937628 branch: octopus description: rbd/mirror/{base/install.yaml clients/{mirror-extra.yaml mirror.yaml} cluster/{2-node.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-low-osd-mem-target.yaml supported-random-distro$/{ubuntu_latest.yaml} workloads/rbd-mirror-snapshot-workunit-journaling.yaml} email: ceph-qa@ceph.io first_in_suite: false job_id: '4937628' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi meta: - desc: run two ceph clusters and install rbd-mirror - desc: configure the permissions for client.mirror - desc: configure the permissions for client.mirror - desc: 2 ceph clusters with 1 mon and 3 osds each - desc: run the rbd_mirror_snapshot.sh workunit to test the rbd-mirror daemon name: teuthology-2020-04-09_06:00:03-rbd-octopus-distro-basic-smithi nuke-on-error: true openstack: - volumes: count: 3 size: 30 os_type: ubuntu os_version: '18.04' overrides: admin_socket: branch: octopus ceph: conf: client: debug rbd: 20 debug rbd_mirror: 15 log to stderr: false rbd default features: 125 client.mirror.0: admin socket: /var/run/ceph/rbd-mirror.$cluster-$name.asok pid file: /var/run/ceph/rbd-mirror.$cluster-$name.pid client.mirror.1: admin socket: /var/run/ceph/rbd-mirror.$cluster-$name.asok pid file: /var/run/ceph/rbd-mirror.$cluster-$name.pid client.mirror.2: admin socket: /var/run/ceph/rbd-mirror.$cluster-$name.asok pid file: /var/run/ceph/rbd-mirror.$cluster-$name.pid client.mirror.3: admin socket: /var/run/ceph/rbd-mirror.$cluster-$name.asok pid file: /var/run/ceph/rbd-mirror.$cluster-$name.pid client.mirror.4: admin socket: /var/run/ceph/rbd-mirror.$cluster-$name.asok pid file: /var/run/ceph/rbd-mirror.$cluster-$name.pid client.mirror.5: admin socket: /var/run/ceph/rbd-mirror.$cluster-$name.asok pid file: /var/run/ceph/rbd-mirror.$cluster-$name.pid client.mirror.6: admin socket: /var/run/ceph/rbd-mirror.$cluster-$name.asok pid file: /var/run/ceph/rbd-mirror.$cluster-$name.pid global: mon client directed command retry: 5 ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug filestore: 20 debug journal: 20 debug ms: 20 debug osd: 25 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd memory target: 1610612736 osd objectstore: bluestore fs: xfs log-whitelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME sha1: 39357d2a782a02d1ea09c76cbe7bc6f222a1127a ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 install: ceph: sha1: 39357d2a782a02d1ea09c76cbe7bc6f222a1127a thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: octopus sha1: 39357d2a782a02d1ea09c76cbe7bc6f222a1127a owner: scheduled_teuthology@teuthology priority: 1000 repo: git://git.ceph.com/ceph.git roles: - - cluster1.mon.a - cluster1.mgr.x - cluster2.mgr.x - cluster1.osd.0 - cluster1.osd.1 - cluster1.osd.2 - cluster1.client.0 - cluster2.client.0 - - cluster2.mon.a - cluster2.osd.0 - cluster2.osd.1 - cluster2.osd.2 - cluster1.client.mirror - cluster1.client.mirror.0 - cluster1.client.mirror.1 - cluster1.client.mirror.2 - cluster1.client.mirror.3 - cluster1.client.mirror.4 - cluster1.client.mirror.5 - cluster1.client.mirror.6 - cluster2.client.mirror - cluster2.client.mirror.0 - cluster2.client.mirror.1 - cluster2.client.mirror.2 - cluster2.client.mirror.3 - cluster2.client.mirror.4 - cluster2.client.mirror.5 - cluster2.client.mirror.6 sha1: 39357d2a782a02d1ea09c76cbe7bc6f222a1127a suite: rbd suite_branch: octopus suite_path: /home/teuthworker/src/git.ceph.com_ceph_octopus/qa suite_relpath: qa suite_repo: git://git.ceph.com/ceph.git suite_sha1: 39357d2a782a02d1ea09c76cbe7bc6f222a1127a tasks: - install: extra_packages: - rbd-mirror - ceph: cluster: cluster1 - ceph: cluster: cluster2 - exec: cluster1.client.mirror.0: - sudo ceph --cluster cluster1 auth caps client.mirror.4 mon 'profile rbd-mirror' osd 'profile rbd' - sudo ceph --cluster cluster1 auth caps client.mirror.5 mon 'profile rbd-mirror' osd 'profile rbd' - sudo ceph --cluster cluster1 auth caps client.mirror.6 mon 'profile rbd-mirror' osd 'profile rbd' - sudo ceph --cluster cluster2 auth caps client.mirror.4 mon 'profile rbd-mirror' osd 'profile rbd' - sudo ceph --cluster cluster2 auth caps client.mirror.5 mon 'profile rbd-mirror' osd 'profile rbd' - sudo ceph --cluster cluster2 auth caps client.mirror.6 mon 'profile rbd-mirror' osd 'profile rbd' - exec: cluster1.client.mirror.0: - sudo ceph --cluster cluster1 auth caps client.mirror mon 'profile rbd-mirror-peer' osd 'profile rbd' - sudo ceph --cluster cluster1 auth caps client.mirror.0 mon 'profile rbd-mirror' osd 'profile rbd' - sudo ceph --cluster cluster1 auth caps client.mirror.1 mon 'profile rbd-mirror' osd 'profile rbd' - sudo ceph --cluster cluster1 auth caps client.mirror.2 mon 'profile rbd-mirror' osd 'profile rbd' - sudo ceph --cluster cluster1 auth caps client.mirror.3 mon 'profile rbd-mirror' osd 'profile rbd' - sudo ceph --cluster cluster2 auth caps client.mirror mon 'profile rbd-mirror-peer' osd 'profile rbd' - sudo ceph --cluster cluster2 auth caps client.mirror.0 mon 'profile rbd-mirror' osd 'profile rbd' - sudo ceph --cluster cluster2 auth caps client.mirror.1 mon 'profile rbd-mirror' osd 'profile rbd' - sudo ceph --cluster cluster2 auth caps client.mirror.2 mon 'profile rbd-mirror' osd 'profile rbd' - sudo ceph --cluster cluster2 auth caps client.mirror.3 mon 'profile rbd-mirror' osd 'profile rbd' - workunit: clients: cluster1.client.mirror: - rbd/rbd_mirror_snapshot.sh env: CEPH_ARGS: '' RBD_IMAGE_FEATURES: layering,exclusive-lock,journaling RBD_MIRROR_CONFIG_KEY: '1' RBD_MIRROR_INSTANCES: '4' RBD_MIRROR_USE_EXISTING_CLUSTER: '1' teuthology_branch: master tube: smithi verbose: true worker_log: /home/teuthworker/archive/worker_logs/worker.smithi.2404 2020-04-10T06:49:20.792 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_octopus/qa; will attempt to use it 2020-04-10T06:49:20.793 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_octopus/qa/tasks 2020-04-10T06:49:20.793 INFO:teuthology.run_tasks:Running task internal.check_packages... 2020-04-10T06:49:20.798 INFO:teuthology.task.internal:Checking packages... 2020-04-10T06:49:20.819 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'basic' and ceph hash '39357d2a782a02d1ea09c76cbe7bc6f222a1127a' 2020-04-10T06:49:20.819 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2020-04-10T06:49:20.819 INFO:teuthology.packaging:ref: None 2020-04-10T06:49:20.819 INFO:teuthology.packaging:tag: None 2020-04-10T06:49:20.819 INFO:teuthology.packaging:branch: octopus 2020-04-10T06:49:20.820 INFO:teuthology.packaging:sha1: 39357d2a782a02d1ea09c76cbe7bc6f222a1127a 2020-04-10T06:49:20.820 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&ref=octopus 2020-04-10T06:49:21.207 INFO:teuthology.task.internal:Found packages for ceph version 15.2.1-1bionic 2020-04-10T06:49:21.208 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2020-04-10T06:49:21.226 INFO:teuthology.task.internal:no buildpackages task found 2020-04-10T06:49:21.227 INFO:teuthology.run_tasks:Running task internal.lock_machines... 2020-04-10T06:49:21.271 INFO:teuthology.task.internal.lock_machines:Locking machines... 2020-04-10T06:49:21.272 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-04-10T06:49:21.322 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-04-10T06:49:31.349 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-04-10T06:49:41.372 DEBUG:teuthology.lock.ops:lock_many request: {'count': 2, 'locked_by': 'scheduled_teuthology@teuthology', 'description': '/home/teuthworker/archive/teuthology-2020-04-09_06:00:03-rbd-octopus-distro-basic-smithi/4937628', 'machine_type': 'smithi'} 2020-04-10T06:49:41.399 DEBUG:teuthology.lock.ops:locked ubuntu@smithi204.front.sepia.ceph.com, ubuntu@smithi101.front.sepia.ceph.com 2020-04-10T06:49:41.489 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=, signals=[15]) 2020-04-10T06:49:41.710 INFO:teuthology.provision.fog.smithi101:Scheduling deploy of ubuntu 18.04 2020-04-10T06:49:41.713 INFO:teuthology.provision.fog.smithi204:Scheduling deploy of ubuntu 18.04 2020-04-10T06:49:41.999 INFO:teuthology.orchestra.console:Power off smithi101 2020-04-10T06:49:41.999 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi101.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2020-04-10T06:49:42.020 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi101.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-04-10T06:49:42.053 INFO:teuthology.orchestra.console:Power off smithi204 2020-04-10T06:49:42.053 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2020-04-10T06:49:42.079 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-04-10T06:49:42.165 INFO:teuthology.orchestra.console:Power off for smithi101 completed 2020-04-10T06:49:42.200 INFO:teuthology.orchestra.console:Power off for smithi204 completed 2020-04-10T06:49:42.266 INFO:teuthology.orchestra.console:Power on smithi101 2020-04-10T06:49:42.267 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi101.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2020-04-10T06:49:42.289 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi101.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-04-10T06:49:42.303 INFO:teuthology.orchestra.console:Power on smithi204 2020-04-10T06:49:42.304 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2020-04-10T06:49:42.328 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-04-10T06:49:46.318 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi101.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-04-10T06:49:46.355 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-04-10T06:49:46.535 INFO:teuthology.orchestra.console:Power on for smithi101 completed 2020-04-10T06:49:46.636 INFO:teuthology.provision.fog.smithi101:Waiting for deploy to finish 2020-04-10T06:49:50.469 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-04-10T06:49:50.680 INFO:teuthology.orchestra.console:Power on for smithi204 completed 2020-04-10T06:49:50.780 INFO:teuthology.provision.fog.smithi204:Waiting for deploy to finish 2020-04-10T06:52:52.185 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi204.front.sepia.ceph.com', 'timeout': 60} 2020-04-10T06:53:01.260 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi204.front.sepia.ceph.com', 'timeout': 60} 2020-04-10T06:53:03.175 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi101.front.sepia.ceph.com', 'timeout': 60} 2020-04-10T06:53:10.343 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi204.front.sepia.ceph.com', 'timeout': 60} 2020-04-10T06:53:12.230 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi101.front.sepia.ceph.com', 'timeout': 60} 2020-04-10T06:53:19.406 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi204.front.sepia.ceph.com', 'timeout': 60} 2020-04-10T06:53:21.323 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi101.front.sepia.ceph.com', 'timeout': 60} 2020-04-10T06:53:28.486 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi204.front.sepia.ceph.com', 'timeout': 60} 2020-04-10T06:53:30.374 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi101.front.sepia.ceph.com', 'timeout': 60} 2020-04-10T06:53:37.577 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi204.front.sepia.ceph.com', 'timeout': 60} 2020-04-10T06:53:39.430 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi101.front.sepia.ceph.com', 'timeout': 60} 2020-04-10T06:53:46.662 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi204.front.sepia.ceph.com', 'timeout': 60} 2020-04-10T06:53:48.489 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi101.front.sepia.ceph.com', 'timeout': 60} 2020-04-10T06:53:55.729 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi204.front.sepia.ceph.com', 'timeout': 60} 2020-04-10T06:53:57.549 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi101.front.sepia.ceph.com', 'timeout': 60} 2020-04-10T06:54:06.637 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi101.front.sepia.ceph.com', 'timeout': 60} 2020-04-10T06:54:27.366 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2020-04-10T06:54:30.987 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi101.front.sepia.ceph.com', 'timeout': 60} 2020-04-10T06:54:33.369 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi204.front.sepia.ceph.com', 'timeout': 60} 2020-04-10T06:54:39.389 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi204.front.sepia.ceph.com', 'timeout': 60} 2020-04-10T06:54:39.400 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2020-04-10T06:54:40.041 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi101.front.sepia.ceph.com', 'timeout': 60} 2020-04-10T06:54:40.047 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2020-04-10T06:54:45.401 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi204.front.sepia.ceph.com', 'timeout': 60} 2020-04-10T06:54:45.411 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2020-04-10T06:54:46.049 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi101.front.sepia.ceph.com', 'timeout': 60} 2020-04-10T06:54:46.059 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2020-04-10T06:54:51.414 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi204.front.sepia.ceph.com', 'timeout': 60} 2020-04-10T06:54:51.425 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2020-04-10T06:54:52.060 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi101.front.sepia.ceph.com', 'timeout': 60} 2020-04-10T06:54:52.070 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2020-04-10T06:54:57.426 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi204.front.sepia.ceph.com', 'timeout': 60} 2020-04-10T06:54:57.499 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T06:54:57.862 INFO:teuthology.orchestra.run.smithi204:> hostname 2020-04-10T06:54:57.908 INFO:teuthology.orchestra.run.smithi204.stdout:smithi204.front.sepia.ceph.com 2020-04-10T06:54:57.908 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T06:54:57.957 INFO:teuthology.orchestra.run.smithi204:> grep smithi204.front.sepia.ceph.com /etc/hosts 2020-04-10T06:54:58.004 INFO:teuthology.orchestra.run.smithi204.stdout:172.21.15.204 smithi204.front.sepia.ceph.com smithi204 2020-04-10T06:54:58.005 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T06:54:58.051 INFO:teuthology.orchestra.run.smithi204:> sudo hostname smithi204 2020-04-10T06:54:58.071 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi101.front.sepia.ceph.com', 'timeout': 60} 2020-04-10T06:54:58.084 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2020-04-10T06:54:58.134 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T06:54:58.179 INFO:teuthology.orchestra.run.smithi204:> sudo sed -i -e 's/smithi204.front.sepia.ceph.com/smithi204/g' /etc/hosts 2020-04-10T06:54:58.239 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T06:54:58.288 INFO:teuthology.orchestra.run.smithi204:> sudo sed -i -e 's/172.21.15.204/172.21.15.204/g' /etc/hosts 2020-04-10T06:54:58.345 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T06:54:58.391 INFO:teuthology.orchestra.run.smithi204:> sudo sed -i -e 's/smithi204.front.sepia.ceph.com/smithi204/g' /etc/hostname 2020-04-10T06:54:58.448 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T06:54:58.495 INFO:teuthology.orchestra.run.smithi204:> sudo hostname smithi204 2020-04-10T06:54:58.547 INFO:teuthology.provision.fog.smithi204:Deploy complete! 2020-04-10T06:54:58.549 DEBUG:teuthology.parallel:result is None 2020-04-10T06:55:04.088 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi101.front.sepia.ceph.com', 'timeout': 60} 2020-04-10T06:55:04.098 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2020-04-10T06:55:10.101 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi101.front.sepia.ceph.com', 'timeout': 60} 2020-04-10T06:55:10.178 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T06:55:10.616 INFO:teuthology.orchestra.run.smithi101:> hostname 2020-04-10T06:55:10.662 INFO:teuthology.orchestra.run.smithi101.stdout:smithi101.front.sepia.ceph.com 2020-04-10T06:55:10.662 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T06:55:10.710 INFO:teuthology.orchestra.run.smithi101:> grep smithi101.front.sepia.ceph.com /etc/hosts 2020-04-10T06:55:10.759 INFO:teuthology.orchestra.run.smithi101.stdout:172.21.15.101 smithi101.front.sepia.ceph.com smithi101 2020-04-10T06:55:10.759 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T06:55:10.809 INFO:teuthology.orchestra.run.smithi101:> sudo hostname smithi101 2020-04-10T06:55:10.874 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T06:55:10.918 INFO:teuthology.orchestra.run.smithi101:> sudo sed -i -e 's/smithi101.front.sepia.ceph.com/smithi101/g' /etc/hosts 2020-04-10T06:55:10.977 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T06:55:11.026 INFO:teuthology.orchestra.run.smithi101:> sudo sed -i -e 's/172.21.15.101/172.21.15.101/g' /etc/hosts 2020-04-10T06:55:11.083 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T06:55:11.131 INFO:teuthology.orchestra.run.smithi101:> sudo sed -i -e 's/smithi101.front.sepia.ceph.com/smithi101/g' /etc/hostname 2020-04-10T06:55:11.189 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T06:55:11.235 INFO:teuthology.orchestra.run.smithi101:> sudo hostname smithi101 2020-04-10T06:55:11.286 INFO:teuthology.provision.fog.smithi101:Deploy complete! 2020-04-10T06:55:11.288 DEBUG:teuthology.parallel:result is None 2020-04-10T06:55:11.420 INFO:teuthology.lock.ops:Checking smithi101.front.sepia.ceph.com 2020-04-10T06:55:11.421 INFO:teuthology.lock.ops:New key found. Updating... 2020-04-10T06:55:11.437 INFO:teuthology.lock.ops:Checking smithi204.front.sepia.ceph.com 2020-04-10T06:55:11.437 INFO:teuthology.lock.ops:New key found. Updating... 2020-04-10T06:55:11.451 INFO:teuthology.lock.ops:Updating [smithi101.front.sepia.ceph.com]: set os type and version on server 2020-04-10T06:55:11.452 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi101.front.sepia.ceph.com', 'timeout': 60} 2020-04-10T06:55:11.563 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T06:55:11.758 INFO:teuthology.orchestra.run.smithi101:> uname -m 2020-04-10T06:55:11.804 INFO:teuthology.orchestra.run.smithi101.stdout:x86_64 2020-04-10T06:55:11.804 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T06:55:11.851 INFO:teuthology.orchestra.run.smithi101:> cat /etc/os-release 2020-04-10T06:55:11.900 INFO:teuthology.orchestra.run.smithi101.stdout:NAME="Ubuntu" 2020-04-10T06:55:11.900 INFO:teuthology.orchestra.run.smithi101.stdout:VERSION="18.04.4 LTS (Bionic Beaver)" 2020-04-10T06:55:11.900 INFO:teuthology.orchestra.run.smithi101.stdout:ID=ubuntu 2020-04-10T06:55:11.900 INFO:teuthology.orchestra.run.smithi101.stdout:ID_LIKE=debian 2020-04-10T06:55:11.900 INFO:teuthology.orchestra.run.smithi101.stdout:PRETTY_NAME="Ubuntu 18.04.4 LTS" 2020-04-10T06:55:11.900 INFO:teuthology.orchestra.run.smithi101.stdout:VERSION_ID="18.04" 2020-04-10T06:55:11.901 INFO:teuthology.orchestra.run.smithi101.stdout:HOME_URL="https://www.ubuntu.com/" 2020-04-10T06:55:11.901 INFO:teuthology.orchestra.run.smithi101.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2020-04-10T06:55:11.901 INFO:teuthology.orchestra.run.smithi101.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2020-04-10T06:55:11.901 INFO:teuthology.orchestra.run.smithi101.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2020-04-10T06:55:11.901 INFO:teuthology.orchestra.run.smithi101.stdout:VERSION_CODENAME=bionic 2020-04-10T06:55:11.901 INFO:teuthology.orchestra.run.smithi101.stdout:UBUNTU_CODENAME=bionic 2020-04-10T06:55:11.902 INFO:teuthology.lock.ops:Updating smithi101.front.sepia.ceph.com on lock server 2020-04-10T06:55:11.925 INFO:teuthology.lock.ops:Updating [smithi204.front.sepia.ceph.com]: set os type and version on server 2020-04-10T06:55:11.925 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi204.front.sepia.ceph.com', 'timeout': 60} 2020-04-10T06:55:12.000 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T06:55:12.240 INFO:teuthology.orchestra.run.smithi204:> uname -m 2020-04-10T06:55:12.289 INFO:teuthology.orchestra.run.smithi204.stdout:x86_64 2020-04-10T06:55:12.289 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T06:55:12.336 INFO:teuthology.orchestra.run.smithi204:> cat /etc/os-release 2020-04-10T06:55:12.385 INFO:teuthology.orchestra.run.smithi204.stdout:NAME="Ubuntu" 2020-04-10T06:55:12.385 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION="18.04.4 LTS (Bionic Beaver)" 2020-04-10T06:55:12.385 INFO:teuthology.orchestra.run.smithi204.stdout:ID=ubuntu 2020-04-10T06:55:12.385 INFO:teuthology.orchestra.run.smithi204.stdout:ID_LIKE=debian 2020-04-10T06:55:12.386 INFO:teuthology.orchestra.run.smithi204.stdout:PRETTY_NAME="Ubuntu 18.04.4 LTS" 2020-04-10T06:55:12.386 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION_ID="18.04" 2020-04-10T06:55:12.386 INFO:teuthology.orchestra.run.smithi204.stdout:HOME_URL="https://www.ubuntu.com/" 2020-04-10T06:55:12.386 INFO:teuthology.orchestra.run.smithi204.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2020-04-10T06:55:12.386 INFO:teuthology.orchestra.run.smithi204.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2020-04-10T06:55:12.386 INFO:teuthology.orchestra.run.smithi204.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2020-04-10T06:55:12.386 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION_CODENAME=bionic 2020-04-10T06:55:12.387 INFO:teuthology.orchestra.run.smithi204.stdout:UBUNTU_CODENAME=bionic 2020-04-10T06:55:12.387 INFO:teuthology.lock.ops:Updating smithi204.front.sepia.ceph.com on lock server 2020-04-10T06:55:12.404 INFO:teuthology.task.internal.lock_machines:2 smithi machines locked this try, 2/2 locked so far 2020-04-10T06:55:12.436 INFO:teuthology.task.internal.lock_machines:Locked targets: smithi101.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDT9Fklxr7Y5E4UHPwH/a7MBIWYJNi0zKSWKVDcl9NGrTUcSbyXl/q3mNS6XQWwWcfHi4kllR1oWNJMGjcjNMqN2Ik7xQG0CN3j6IxR/ThSBv7eyIUSAldogJUNIte/ae3MyZlpyKhMSxSII/tStu2kQ4lchx7uAYYoXPW8dg178Lr3m/alXSFlfu5TrZIhfa7lKMmYXhXrrkwWDaaX8jqpXi+SYBkAa3mWhHiMihALIKdQRsUmMaAlPYQSnVuA4iU4JIt3x9/0BXHIozs8hrUMQ7MLSjXaKmH3trGj8P/hy65A7yVd1F4RUMUUHcK4AVo/4gHlW/G3+y7vOtBUfDXJ smithi204.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCSzeRmp/sjnId4Ox3+K4SEbMVfVJu6es2vZjpBuMNFfBvRZF3JaOt+F82hMD1lDOJFSzAqT/hKwcdqT5vMPkbkQVnTUTz6KWx5h0Magh8oRggNmVvOswW8Gcz8ktHBV+GSTWy8DISCgruSYDn2z57IoqFYwdf0GFBhZMX1RNs4nqCIHW3RKLCtRlD6g9P2wHktAjGcJk73J8rY66EVc4+hY/V6p67qXU1jU2mCc0ktr4U2vgIRYS4p4AZi1uSYG934vbZTMzvm4YFFt1M6ygHYNPgxF77Z22HIti6AGi3lx0S84WT2wY2NnlAbTeMzvJa8eZnARRhSv+fYJPGVcz+r 2020-04-10T06:55:12.436 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-04-10T06:55:12.491 INFO:teuthology.run_tasks:Running task internal.save_config... 2020-04-10T06:55:12.503 INFO:teuthology.task.internal:Saving configuration 2020-04-10T06:55:12.531 INFO:teuthology.run_tasks:Running task internal.check_lock... 2020-04-10T06:55:12.542 INFO:teuthology.task.internal.check_lock:Checking locks... 2020-04-10T06:55:12.559 DEBUG:teuthology.task.internal.check_lock:machine status is {u'is_vm': False, u'locked': True, u'locked_since': u'2020-04-10 06:49:41.388612', u'locked_by': u'scheduled_teuthology@teuthology', u'up': True, u'mac_address': None, u'name': u'smithi101.front.sepia.ceph.com', u'os_version': u'18.04', u'machine_type': u'smithi', u'vm_host': None, u'os_type': u'ubuntu', u'arch': u'x86_64', u'ssh_pub_key': u'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIM5fx+c4+03aU8txWO2j55w/dzIwGwNWjrlGrKsIBnA2', u'description': u'/home/teuthworker/archive/teuthology-2020-04-09_06:00:03-rbd-octopus-distro-basic-smithi/4937628'} 2020-04-10T06:55:12.574 DEBUG:teuthology.task.internal.check_lock:machine status is {u'is_vm': False, u'locked': True, u'locked_since': u'2020-04-10 06:49:41.388589', u'locked_by': u'scheduled_teuthology@teuthology', u'up': True, u'mac_address': None, u'name': u'smithi204.front.sepia.ceph.com', u'os_version': u'18.04', u'machine_type': u'smithi', u'vm_host': None, u'os_type': u'ubuntu', u'arch': u'x86_64', u'ssh_pub_key': u'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIACMEw3H+EwHO3UOFkNAnAofwmwbn3xFsyAm0l5kS1Qo', u'description': u'/home/teuthworker/archive/teuthology-2020-04-09_06:00:03-rbd-octopus-distro-basic-smithi/4937628'} 2020-04-10T06:55:12.574 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2020-04-10T06:55:12.585 INFO:teuthology.task.internal:roles: ubuntu@smithi101.front.sepia.ceph.com - ['cluster1.mon.a', 'cluster1.mgr.x', 'cluster2.mgr.x', 'cluster1.osd.0', 'cluster1.osd.1', 'cluster1.osd.2', 'cluster1.client.0', 'cluster2.client.0'] 2020-04-10T06:55:12.585 INFO:teuthology.task.internal:roles: ubuntu@smithi204.front.sepia.ceph.com - ['cluster2.mon.a', 'cluster2.osd.0', 'cluster2.osd.1', 'cluster2.osd.2', 'cluster1.client.mirror', 'cluster1.client.mirror.0', 'cluster1.client.mirror.1', 'cluster1.client.mirror.2', 'cluster1.client.mirror.3', 'cluster1.client.mirror.4', 'cluster1.client.mirror.5', 'cluster1.client.mirror.6', 'cluster2.client.mirror', 'cluster2.client.mirror.0', 'cluster2.client.mirror.1', 'cluster2.client.mirror.2', 'cluster2.client.mirror.3', 'cluster2.client.mirror.4', 'cluster2.client.mirror.5', 'cluster2.client.mirror.6'] 2020-04-10T06:55:12.585 INFO:teuthology.run_tasks:Running task console_log... 2020-04-10T06:55:12.658 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=, signals=[15]) 2020-04-10T06:55:12.658 INFO:teuthology.run_tasks:Running task internal.connect... 2020-04-10T06:55:12.669 INFO:teuthology.task.internal:Opening connections... 2020-04-10T06:55:12.670 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi101.front.sepia.ceph.com 2020-04-10T06:55:12.671 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi101.front.sepia.ceph.com', 'timeout': 60} 2020-04-10T06:55:12.746 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi204.front.sepia.ceph.com 2020-04-10T06:55:12.747 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi204.front.sepia.ceph.com', 'timeout': 60} 2020-04-10T06:55:12.824 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2020-04-10T06:55:12.837 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T06:55:12.938 INFO:teuthology.orchestra.run.smithi101:> uname -m 2020-04-10T06:55:12.985 INFO:teuthology.orchestra.run.smithi101.stdout:x86_64 2020-04-10T06:55:12.985 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T06:55:13.032 INFO:teuthology.orchestra.run.smithi101:> cat /etc/os-release 2020-04-10T06:55:13.080 INFO:teuthology.orchestra.run.smithi101.stdout:NAME="Ubuntu" 2020-04-10T06:55:13.081 INFO:teuthology.orchestra.run.smithi101.stdout:VERSION="18.04.4 LTS (Bionic Beaver)" 2020-04-10T06:55:13.081 INFO:teuthology.orchestra.run.smithi101.stdout:ID=ubuntu 2020-04-10T06:55:13.081 INFO:teuthology.orchestra.run.smithi101.stdout:ID_LIKE=debian 2020-04-10T06:55:13.081 INFO:teuthology.orchestra.run.smithi101.stdout:PRETTY_NAME="Ubuntu 18.04.4 LTS" 2020-04-10T06:55:13.081 INFO:teuthology.orchestra.run.smithi101.stdout:VERSION_ID="18.04" 2020-04-10T06:55:13.081 INFO:teuthology.orchestra.run.smithi101.stdout:HOME_URL="https://www.ubuntu.com/" 2020-04-10T06:55:13.081 INFO:teuthology.orchestra.run.smithi101.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2020-04-10T06:55:13.082 INFO:teuthology.orchestra.run.smithi101.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2020-04-10T06:55:13.082 INFO:teuthology.orchestra.run.smithi101.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2020-04-10T06:55:13.082 INFO:teuthology.orchestra.run.smithi101.stdout:VERSION_CODENAME=bionic 2020-04-10T06:55:13.082 INFO:teuthology.orchestra.run.smithi101.stdout:UBUNTU_CODENAME=bionic 2020-04-10T06:55:13.082 INFO:teuthology.lock.ops:Updating smithi101.front.sepia.ceph.com on lock server 2020-04-10T06:55:13.097 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T06:55:13.105 INFO:teuthology.orchestra.run.smithi204:> uname -m 2020-04-10T06:55:13.152 INFO:teuthology.orchestra.run.smithi204.stdout:x86_64 2020-04-10T06:55:13.153 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T06:55:13.199 INFO:teuthology.orchestra.run.smithi204:> cat /etc/os-release 2020-04-10T06:55:13.248 INFO:teuthology.orchestra.run.smithi204.stdout:NAME="Ubuntu" 2020-04-10T06:55:13.248 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION="18.04.4 LTS (Bionic Beaver)" 2020-04-10T06:55:13.249 INFO:teuthology.orchestra.run.smithi204.stdout:ID=ubuntu 2020-04-10T06:55:13.249 INFO:teuthology.orchestra.run.smithi204.stdout:ID_LIKE=debian 2020-04-10T06:55:13.249 INFO:teuthology.orchestra.run.smithi204.stdout:PRETTY_NAME="Ubuntu 18.04.4 LTS" 2020-04-10T06:55:13.249 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION_ID="18.04" 2020-04-10T06:55:13.249 INFO:teuthology.orchestra.run.smithi204.stdout:HOME_URL="https://www.ubuntu.com/" 2020-04-10T06:55:13.249 INFO:teuthology.orchestra.run.smithi204.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2020-04-10T06:55:13.250 INFO:teuthology.orchestra.run.smithi204.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2020-04-10T06:55:13.250 INFO:teuthology.orchestra.run.smithi204.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2020-04-10T06:55:13.250 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION_CODENAME=bionic 2020-04-10T06:55:13.250 INFO:teuthology.orchestra.run.smithi204.stdout:UBUNTU_CODENAME=bionic 2020-04-10T06:55:13.250 INFO:teuthology.lock.ops:Updating smithi204.front.sepia.ceph.com on lock server 2020-04-10T06:55:13.264 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2020-04-10T06:55:13.313 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2020-04-10T06:55:13.323 INFO:teuthology.task.internal:Checking for old test directory... 2020-04-10T06:55:13.323 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T06:55:13.338 INFO:teuthology.orchestra.run.smithi101:> test '!' -e /home/ubuntu/cephtest 2020-04-10T06:55:13.381 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T06:55:13.386 INFO:teuthology.orchestra.run.smithi204:> test '!' -e /home/ubuntu/cephtest 2020-04-10T06:55:13.436 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2020-04-10T06:55:13.456 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2020-04-10T06:55:13.456 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T06:55:13.474 INFO:teuthology.orchestra.run.smithi101:> test -z $(ls -A /var/lib/ceph) 2020-04-10T06:55:13.523 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T06:55:13.529 INFO:teuthology.orchestra.run.smithi204:> test -z $(ls -A /var/lib/ceph) 2020-04-10T06:55:13.589 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2020-04-10T06:55:13.630 INFO:teuthology.run_tasks:Running task kernel... 2020-04-10T06:55:13.648 DEBUG:teuthology.task.kernel:normalized config {'cluster2.mgr.x': {'sha1': 'distro', 'kdb': True}, 'cluster1.mon.a': {'sha1': 'distro', 'kdb': True}, 'cluster2.client.mirror.4': {'sha1': 'distro', 'kdb': True}, 'cluster2.client.mirror.5': {'sha1': 'distro', 'kdb': True}, 'cluster2.client.mirror.6': {'sha1': 'distro', 'kdb': True}, 'cluster2.client.mirror.0': {'sha1': 'distro', 'kdb': True}, 'cluster2.client.mirror.1': {'sha1': 'distro', 'kdb': True}, 'cluster2.client.mirror.2': {'sha1': 'distro', 'kdb': True}, 'cluster2.client.mirror.3': {'sha1': 'distro', 'kdb': True}, 'cluster1.client.mirror.6': {'sha1': 'distro', 'kdb': True}, 'cluster1.client.mirror.5': {'sha1': 'distro', 'kdb': True}, 'cluster1.client.mirror.4': {'sha1': 'distro', 'kdb': True}, 'cluster1.client.mirror.3': {'sha1': 'distro', 'kdb': True}, 'cluster1.client.mirror.2': {'sha1': 'distro', 'kdb': True}, 'cluster1.client.mirror.1': {'sha1': 'distro', 'kdb': True}, 'cluster1.client.mirror.0': {'sha1': 'distro', 'kdb': True}, 'cluster2.client.mirror': {'sha1': 'distro', 'kdb': True}, 'cluster1.client.mirror': {'sha1': 'distro', 'kdb': True}, 'cluster1.osd.2': {'sha1': 'distro', 'kdb': True}, 'cluster1.osd.1': {'sha1': 'distro', 'kdb': True}, 'cluster1.osd.0': {'sha1': 'distro', 'kdb': True}, 'cluster2.osd.1': {'sha1': 'distro', 'kdb': True}, 'cluster1.client.0': {'sha1': 'distro', 'kdb': True}, 'cluster2.client.0': {'sha1': 'distro', 'kdb': True}, 'cluster2.osd.0': {'sha1': 'distro', 'kdb': True}, 'cluster2.mon.a': {'sha1': 'distro', 'kdb': True}, 'cluster2.osd.2': {'sha1': 'distro', 'kdb': True}, 'cluster1.mgr.x': {'sha1': 'distro', 'kdb': True}} 2020-04-10T06:55:13.648 INFO:teuthology.task.kernel:config {'cluster1.mon.a': {'sha1': 'distro', 'kdb': True}, 'cluster2.mon.a': {'sha1': 'distro', 'kdb': True}}, timeout 300 2020-04-10T06:55:13.648 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T06:55:13.654 INFO:teuthology.orchestra.run.smithi101:> uname -r 2020-04-10T06:55:13.702 INFO:teuthology.orchestra.run.smithi101.stdout:4.15.0-91-generic 2020-04-10T06:55:13.702 INFO:teuthology.task.kernel:Running kernel on smithi101: 4.15.0-91-generic 2020-04-10T06:55:13.702 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T06:55:13.752 INFO:teuthology.orchestra.run.smithi101:> sudo apt-get clean 2020-04-10T06:55:13.860 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T06:55:13.866 INFO:teuthology.orchestra.run.smithi101:> sudo apt-get update 2020-04-10T06:55:14.192 INFO:teuthology.orchestra.run.smithi101.stdout:Hit:1 http://archive.ubuntu.com/ubuntu bionic InRelease 2020-04-10T06:55:14.192 INFO:teuthology.orchestra.run.smithi101.stdout:Get:2 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB] 2020-04-10T06:55:14.260 INFO:teuthology.orchestra.run.smithi101.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB] 2020-04-10T06:55:14.568 INFO:teuthology.orchestra.run.smithi101.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB] 2020-04-10T06:55:14.642 INFO:teuthology.orchestra.run.smithi101.stdout:Get:5 http://security.ubuntu.com/ubuntu bionic-security/main i386 Packages [459 kB] 2020-04-10T06:55:14.772 INFO:teuthology.orchestra.run.smithi101.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [914 kB] 2020-04-10T06:55:14.953 INFO:teuthology.orchestra.run.smithi101.stdout:Get:7 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [692 kB] 2020-04-10T06:55:15.050 INFO:teuthology.orchestra.run.smithi101.stdout:Get:8 http://security.ubuntu.com/ubuntu bionic-security/main Translation-en [221 kB] 2020-04-10T06:55:15.074 INFO:teuthology.orchestra.run.smithi101.stdout:Get:9 http://security.ubuntu.com/ubuntu bionic-security/restricted amd64 Packages [34.3 kB] 2020-04-10T06:55:15.078 INFO:teuthology.orchestra.run.smithi101.stdout:Get:10 http://security.ubuntu.com/ubuntu bionic-security/restricted Translation-en [8,924 B] 2020-04-10T06:55:15.078 INFO:teuthology.orchestra.run.smithi101.stdout:Get:11 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [657 kB] 2020-04-10T06:55:15.131 INFO:teuthology.orchestra.run.smithi101.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic-updates/main i386 Packages [668 kB] 2020-04-10T06:55:15.147 INFO:teuthology.orchestra.run.smithi101.stdout:Get:13 http://security.ubuntu.com/ubuntu bionic-security/universe i386 Packages [618 kB] 2020-04-10T06:55:15.170 INFO:teuthology.orchestra.run.smithi101.stdout:Get:14 http://archive.ubuntu.com/ubuntu bionic-updates/main Translation-en [314 kB] 2020-04-10T06:55:15.176 INFO:teuthology.orchestra.run.smithi101.stdout:Get:15 http://security.ubuntu.com/ubuntu bionic-security/universe Translation-en [218 kB] 2020-04-10T06:55:15.187 INFO:teuthology.orchestra.run.smithi101.stdout:Get:16 http://security.ubuntu.com/ubuntu bionic-security/multiverse amd64 Packages [7,176 B] 2020-04-10T06:55:15.187 INFO:teuthology.orchestra.run.smithi101.stdout:Get:17 http://security.ubuntu.com/ubuntu bionic-security/multiverse Translation-en [2,764 B] 2020-04-10T06:55:15.213 INFO:teuthology.orchestra.run.smithi101.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [43.9 kB] 2020-04-10T06:55:15.215 INFO:teuthology.orchestra.run.smithi101.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic-updates/restricted Translation-en [11.0 kB] 2020-04-10T06:55:15.218 INFO:teuthology.orchestra.run.smithi101.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [1,065 kB] 2020-04-10T06:55:15.276 INFO:teuthology.orchestra.run.smithi101.stdout:Get:21 http://archive.ubuntu.com/ubuntu bionic-updates/universe i386 Packages [1,013 kB] 2020-04-10T06:55:15.311 INFO:teuthology.orchestra.run.smithi101.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic-updates/universe Translation-en [330 kB] 2020-04-10T06:55:15.325 INFO:teuthology.orchestra.run.smithi101.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse amd64 Packages [10.8 kB] 2020-04-10T06:55:15.326 INFO:teuthology.orchestra.run.smithi101.stdout:Get:24 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse Translation-en [4,728 B] 2020-04-10T06:55:15.326 INFO:teuthology.orchestra.run.smithi101.stdout:Get:25 http://archive.ubuntu.com/ubuntu bionic-backports/universe i386 Packages [4,016 B] 2020-04-10T06:55:15.326 INFO:teuthology.orchestra.run.smithi101.stdout:Get:26 http://archive.ubuntu.com/ubuntu bionic-backports/universe amd64 Packages [4,020 B] 2020-04-10T06:55:16.206 INFO:teuthology.orchestra.run.smithi101.stdout:Fetched 7,553 kB in 2s (3,617 kB/s) 2020-04-10T06:55:17.512 INFO:teuthology.orchestra.run.smithi101.stdout:Reading package lists... 2020-04-10T06:55:17.566 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T06:55:17.575 INFO:teuthology.orchestra.run.smithi101:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2020-04-10T06:55:17.703 INFO:teuthology.orchestra.run.smithi101.stdout:Reading package lists... 2020-04-10T06:55:17.949 INFO:teuthology.orchestra.run.smithi101.stdout:Building dependency tree... 2020-04-10T06:55:17.955 INFO:teuthology.orchestra.run.smithi101.stdout:Reading state information... 2020-04-10T06:55:18.024 INFO:teuthology.orchestra.run.smithi101.stderr:E: Unable to locate package linux-image-current-generic 2020-04-10T06:55:18.025 DEBUG:teuthology.orchestra.run:got remote process result: 100 2020-04-10T06:55:18.025 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T06:55:18.031 INFO:teuthology.orchestra.run.smithi101:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2020-04-10T06:55:18.146 INFO:teuthology.orchestra.run.smithi101.stdout:Reading package lists... 2020-04-10T06:55:18.368 INFO:teuthology.orchestra.run.smithi101.stdout:Building dependency tree... 2020-04-10T06:55:18.369 INFO:teuthology.orchestra.run.smithi101.stdout:Reading state information... 2020-04-10T06:55:18.532 INFO:teuthology.orchestra.run.smithi101.stdout:The following additional packages will be installed: 2020-04-10T06:55:18.533 INFO:teuthology.orchestra.run.smithi101.stdout: linux-generic linux-headers-4.15.0-96 linux-headers-4.15.0-96-generic 2020-04-10T06:55:18.533 INFO:teuthology.orchestra.run.smithi101.stdout: linux-headers-generic linux-image-4.15.0-96-generic 2020-04-10T06:55:18.533 INFO:teuthology.orchestra.run.smithi101.stdout: linux-modules-4.15.0-96-generic linux-modules-extra-4.15.0-96-generic 2020-04-10T06:55:18.534 INFO:teuthology.orchestra.run.smithi101.stdout:Suggested packages: 2020-04-10T06:55:18.534 INFO:teuthology.orchestra.run.smithi101.stdout: fdutils linux-doc-4.15.0 | linux-source-4.15.0 linux-tools 2020-04-10T06:55:18.534 INFO:teuthology.orchestra.run.smithi101.stdout:Recommended packages: 2020-04-10T06:55:18.534 INFO:teuthology.orchestra.run.smithi101.stdout: thermald 2020-04-10T06:55:18.567 INFO:teuthology.orchestra.run.smithi101.stdout:The following NEW packages will be installed: 2020-04-10T06:55:18.567 INFO:teuthology.orchestra.run.smithi101.stdout: linux-headers-4.15.0-96 linux-headers-4.15.0-96-generic 2020-04-10T06:55:18.567 INFO:teuthology.orchestra.run.smithi101.stdout: linux-image-4.15.0-96-generic linux-modules-4.15.0-96-generic 2020-04-10T06:55:18.568 INFO:teuthology.orchestra.run.smithi101.stdout: linux-modules-extra-4.15.0-96-generic 2020-04-10T06:55:18.577 INFO:teuthology.orchestra.run.smithi101.stdout:The following packages will be upgraded: 2020-04-10T06:55:18.578 INFO:teuthology.orchestra.run.smithi101.stdout: linux-generic linux-headers-generic linux-image-generic 2020-04-10T06:55:18.750 INFO:teuthology.orchestra.run.smithi101.stdout:3 upgraded, 5 newly installed, 0 to remove and 50 not upgraded. 2020-04-10T06:55:18.750 INFO:teuthology.orchestra.run.smithi101.stdout:Need to get 65.9 MB of archives. 2020-04-10T06:55:18.750 INFO:teuthology.orchestra.run.smithi101.stdout:After this operation, 332 MB of additional disk space will be used. 2020-04-10T06:55:18.750 INFO:teuthology.orchestra.run.smithi101.stdout:Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-modules-4.15.0-96-generic amd64 4.15.0-96.97 [13.1 MB] 2020-04-10T06:55:19.764 INFO:teuthology.orchestra.run.smithi101.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-image-4.15.0-96-generic amd64 4.15.0-96.97 [7,998 kB] 2020-04-10T06:55:19.958 INFO:teuthology.orchestra.run.smithi101.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-modules-extra-4.15.0-96-generic amd64 4.15.0-96.97 [32.8 MB] 2020-04-10T06:55:20.875 INFO:teuthology.orchestra.run.smithi101.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-generic amd64 4.15.0.96.87 [1,868 B] 2020-04-10T06:55:20.875 INFO:teuthology.orchestra.run.smithi101.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-image-generic amd64 4.15.0.96.87 [2,468 B] 2020-04-10T06:55:20.875 INFO:teuthology.orchestra.run.smithi101.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-4.15.0-96 all 4.15.0-96.97 [11.0 MB] 2020-04-10T06:55:21.225 INFO:teuthology.orchestra.run.smithi101.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-4.15.0-96-generic amd64 4.15.0-96.97 [1,127 kB] 2020-04-10T06:55:21.250 INFO:teuthology.orchestra.run.smithi101.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-generic amd64 4.15.0.96.87 [2,432 B] 2020-04-10T06:55:21.763 INFO:teuthology.orchestra.run.smithi101.stdout:Fetched 65.9 MB in 3s (24.7 MB/s) 2020-04-10T06:55:21.912 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package linux-modules-4.15.0-96-generic. 2020-04-10T06:55:21.949 INFO:teuthology.orchestra.run.smithi101.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 101956 files and directories currently installed.) 2020-04-10T06:55:21.952 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../0-linux-modules-4.15.0-96-generic_4.15.0-96.97_amd64.deb ... 2020-04-10T06:55:22.001 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking linux-modules-4.15.0-96-generic (4.15.0-96.97) ... 2020-04-10T06:55:23.553 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package linux-image-4.15.0-96-generic. 2020-04-10T06:55:23.566 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../1-linux-image-4.15.0-96-generic_4.15.0-96.97_amd64.deb ... 2020-04-10T06:55:23.653 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking linux-image-4.15.0-96-generic (4.15.0-96.97) ... 2020-04-10T06:55:24.190 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package linux-modules-extra-4.15.0-96-generic. 2020-04-10T06:55:24.201 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../2-linux-modules-extra-4.15.0-96-generic_4.15.0-96.97_amd64.deb ... 2020-04-10T06:55:24.241 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking linux-modules-extra-4.15.0-96-generic (4.15.0-96.97) ... 2020-04-10T06:55:28.054 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../3-linux-generic_4.15.0.96.87_amd64.deb ... 2020-04-10T06:55:28.173 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking linux-generic (4.15.0.96.87) over (4.15.0.91.83) ... 2020-04-10T06:55:28.498 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../4-linux-image-generic_4.15.0.96.87_amd64.deb ... 2020-04-10T06:55:28.594 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking linux-image-generic (4.15.0.96.87) over (4.15.0.91.83) ... 2020-04-10T06:55:28.913 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package linux-headers-4.15.0-96. 2020-04-10T06:55:28.925 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../5-linux-headers-4.15.0-96_4.15.0-96.97_all.deb ... 2020-04-10T06:55:28.963 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking linux-headers-4.15.0-96 (4.15.0-96.97) ... 2020-04-10T06:55:33.629 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package linux-headers-4.15.0-96-generic. 2020-04-10T06:55:33.644 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../6-linux-headers-4.15.0-96-generic_4.15.0-96.97_amd64.deb ... 2020-04-10T06:55:33.685 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking linux-headers-4.15.0-96-generic (4.15.0-96.97) ... 2020-04-10T06:55:36.122 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../7-linux-headers-generic_4.15.0.96.87_amd64.deb ... 2020-04-10T06:55:36.246 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking linux-headers-generic (4.15.0.96.87) over (4.15.0.91.83) ... 2020-04-10T06:55:36.567 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up linux-modules-4.15.0-96-generic (4.15.0-96.97) ... 2020-04-10T06:55:37.126 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up linux-headers-4.15.0-96 (4.15.0-96.97) ... 2020-04-10T06:55:37.252 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up linux-image-4.15.0-96-generic (4.15.0-96.97) ... 2020-04-10T06:55:37.776 INFO:teuthology.orchestra.run.smithi101.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-4.15.0-96-generic 2020-04-10T06:55:37.776 INFO:teuthology.orchestra.run.smithi101.stdout:I: /initrd.img is now a symlink to boot/initrd.img-4.15.0-96-generic 2020-04-10T06:55:37.931 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up linux-modules-extra-4.15.0-96-generic (4.15.0-96.97) ... 2020-04-10T06:55:38.423 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up linux-headers-4.15.0-96-generic (4.15.0-96.97) ... 2020-04-10T06:55:38.523 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up linux-image-generic (4.15.0.96.87) ... 2020-04-10T06:55:38.649 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up linux-headers-generic (4.15.0.96.87) ... 2020-04-10T06:55:38.776 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up linux-generic (4.15.0.96.87) ... 2020-04-10T06:55:38.901 INFO:teuthology.orchestra.run.smithi101.stdout:Processing triggers for linux-image-4.15.0-96-generic (4.15.0-96.97) ... 2020-04-10T06:55:39.103 INFO:teuthology.orchestra.run.smithi101.stdout:/etc/kernel/postinst.d/initramfs-tools: 2020-04-10T06:55:39.104 INFO:teuthology.orchestra.run.smithi101.stdout:update-initramfs: Generating /boot/initrd.img-4.15.0-96-generic 2020-04-10T06:55:53.460 INFO:teuthology.orchestra.run.smithi101.stdout:/etc/kernel/postinst.d/zz-update-grub: 2020-04-10T06:55:53.461 INFO:teuthology.orchestra.run.smithi101.stdout:Sourcing file `/etc/default/grub' 2020-04-10T06:55:53.484 INFO:teuthology.orchestra.run.smithi101.stdout:Generating grub configuration file ... 2020-04-10T06:55:53.831 INFO:teuthology.orchestra.run.smithi101.stdout:Found linux image: /boot/vmlinuz-4.15.0-96-generic 2020-04-10T06:55:53.873 INFO:teuthology.orchestra.run.smithi101.stdout:Found initrd image: /boot/initrd.img-4.15.0-96-generic 2020-04-10T06:55:54.205 INFO:teuthology.orchestra.run.smithi101.stdout:Found linux image: /boot/vmlinuz-4.15.0-91-generic 2020-04-10T06:55:54.235 INFO:teuthology.orchestra.run.smithi101.stdout:Found initrd image: /boot/initrd.img-4.15.0-91-generic 2020-04-10T06:55:55.745 INFO:teuthology.orchestra.run.smithi101.stdout:done 2020-04-10T06:55:56.126 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T06:55:56.129 INFO:teuthology.orchestra.run.smithi101:> dpkg -s linux-image-generic 2020-04-10T06:55:56.190 INFO:teuthology.orchestra.run.smithi101.stdout:Package: linux-image-generic 2020-04-10T06:55:56.191 INFO:teuthology.orchestra.run.smithi101.stdout:Status: install ok installed 2020-04-10T06:55:56.191 INFO:teuthology.orchestra.run.smithi101.stdout:Priority: optional 2020-04-10T06:55:56.191 INFO:teuthology.orchestra.run.smithi101.stdout:Section: kernel 2020-04-10T06:55:56.191 INFO:teuthology.orchestra.run.smithi101.stdout:Installed-Size: 16 2020-04-10T06:55:56.191 INFO:teuthology.orchestra.run.smithi101.stdout:Maintainer: Ubuntu Kernel Team 2020-04-10T06:55:56.191 INFO:teuthology.orchestra.run.smithi101.stdout:Architecture: amd64 2020-04-10T06:55:56.192 INFO:teuthology.orchestra.run.smithi101.stdout:Source: linux-meta 2020-04-10T06:55:56.192 INFO:teuthology.orchestra.run.smithi101.stdout:Version: 4.15.0.96.87 2020-04-10T06:55:56.192 INFO:teuthology.orchestra.run.smithi101.stdout:Depends: linux-image-4.15.0-96-generic, linux-modules-extra-4.15.0-96-generic, linux-firmware, intel-microcode, amd64-microcode 2020-04-10T06:55:56.192 INFO:teuthology.orchestra.run.smithi101.stdout:Recommends: thermald 2020-04-10T06:55:56.192 INFO:teuthology.orchestra.run.smithi101.stdout:Description: Generic Linux kernel image 2020-04-10T06:55:56.192 INFO:teuthology.orchestra.run.smithi101.stdout: This package will always depend on the latest generic kernel image 2020-04-10T06:55:56.192 INFO:teuthology.orchestra.run.smithi101.stdout: available. 2020-04-10T06:55:56.193 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.15.0-91-generic Expected: 4.15.0-96-generic 2020-04-10T06:55:56.193 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T06:55:56.199 INFO:teuthology.orchestra.run.smithi204:> uname -r 2020-04-10T06:55:56.247 INFO:teuthology.orchestra.run.smithi204.stdout:4.15.0-91-generic 2020-04-10T06:55:56.248 INFO:teuthology.task.kernel:Running kernel on smithi204: 4.15.0-91-generic 2020-04-10T06:55:56.248 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T06:55:56.294 INFO:teuthology.orchestra.run.smithi204:> sudo apt-get clean 2020-04-10T06:55:56.405 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T06:55:56.411 INFO:teuthology.orchestra.run.smithi204:> sudo apt-get update 2020-04-10T06:55:56.711 INFO:teuthology.orchestra.run.smithi204.stdout:Hit:1 http://archive.ubuntu.com/ubuntu bionic InRelease 2020-04-10T06:55:56.711 INFO:teuthology.orchestra.run.smithi204.stdout:Get:2 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB] 2020-04-10T06:55:56.781 INFO:teuthology.orchestra.run.smithi204.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB] 2020-04-10T06:55:57.107 INFO:teuthology.orchestra.run.smithi204.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB] 2020-04-10T06:55:57.177 INFO:teuthology.orchestra.run.smithi204.stdout:Get:5 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [692 kB] 2020-04-10T06:55:57.311 INFO:teuthology.orchestra.run.smithi204.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [914 kB] 2020-04-10T06:55:57.533 INFO:teuthology.orchestra.run.smithi204.stdout:Get:7 http://security.ubuntu.com/ubuntu bionic-security/main i386 Packages [459 kB] 2020-04-10T06:55:57.568 INFO:teuthology.orchestra.run.smithi204.stdout:Get:8 http://security.ubuntu.com/ubuntu bionic-security/main Translation-en [221 kB] 2020-04-10T06:55:57.617 INFO:teuthology.orchestra.run.smithi204.stdout:Get:9 http://security.ubuntu.com/ubuntu bionic-security/restricted amd64 Packages [34.3 kB] 2020-04-10T06:55:57.619 INFO:teuthology.orchestra.run.smithi204.stdout:Get:10 http://security.ubuntu.com/ubuntu bionic-security/restricted Translation-en [8,924 B] 2020-04-10T06:55:57.625 INFO:teuthology.orchestra.run.smithi204.stdout:Get:11 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [657 kB] 2020-04-10T06:55:57.664 INFO:teuthology.orchestra.run.smithi204.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic-updates/main i386 Packages [668 kB] 2020-04-10T06:55:57.681 INFO:teuthology.orchestra.run.smithi204.stdout:Get:13 http://security.ubuntu.com/ubuntu bionic-security/universe i386 Packages [618 kB] 2020-04-10T06:55:57.704 INFO:teuthology.orchestra.run.smithi204.stdout:Get:14 http://archive.ubuntu.com/ubuntu bionic-updates/main Translation-en [314 kB] 2020-04-10T06:55:57.704 INFO:teuthology.orchestra.run.smithi204.stdout:Get:15 http://security.ubuntu.com/ubuntu bionic-security/universe Translation-en [218 kB] 2020-04-10T06:55:57.713 INFO:teuthology.orchestra.run.smithi204.stdout:Get:16 http://security.ubuntu.com/ubuntu bionic-security/multiverse amd64 Packages [7,176 B] 2020-04-10T06:55:57.713 INFO:teuthology.orchestra.run.smithi204.stdout:Get:17 http://security.ubuntu.com/ubuntu bionic-security/multiverse Translation-en [2,764 B] 2020-04-10T06:55:57.753 INFO:teuthology.orchestra.run.smithi204.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [43.9 kB] 2020-04-10T06:55:57.753 INFO:teuthology.orchestra.run.smithi204.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic-updates/restricted Translation-en [11.0 kB] 2020-04-10T06:55:57.754 INFO:teuthology.orchestra.run.smithi204.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [1,065 kB] 2020-04-10T06:55:57.810 INFO:teuthology.orchestra.run.smithi204.stdout:Get:21 http://archive.ubuntu.com/ubuntu bionic-updates/universe i386 Packages [1,013 kB] 2020-04-10T06:55:57.846 INFO:teuthology.orchestra.run.smithi204.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic-updates/universe Translation-en [330 kB] 2020-04-10T06:55:57.855 INFO:teuthology.orchestra.run.smithi204.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse amd64 Packages [10.8 kB] 2020-04-10T06:55:57.855 INFO:teuthology.orchestra.run.smithi204.stdout:Get:24 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse Translation-en [4,728 B] 2020-04-10T06:55:57.856 INFO:teuthology.orchestra.run.smithi204.stdout:Get:25 http://archive.ubuntu.com/ubuntu bionic-backports/universe i386 Packages [4,016 B] 2020-04-10T06:55:57.856 INFO:teuthology.orchestra.run.smithi204.stdout:Get:26 http://archive.ubuntu.com/ubuntu bionic-backports/universe amd64 Packages [4,020 B] 2020-04-10T06:55:58.625 INFO:teuthology.orchestra.run.smithi204.stdout:Fetched 7,553 kB in 2s (3,835 kB/s) 2020-04-10T06:56:00.036 INFO:teuthology.orchestra.run.smithi204.stdout:Reading package lists... 2020-04-10T06:56:00.078 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T06:56:00.089 INFO:teuthology.orchestra.run.smithi204:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2020-04-10T06:56:00.208 INFO:teuthology.orchestra.run.smithi204.stdout:Reading package lists... 2020-04-10T06:56:00.438 INFO:teuthology.orchestra.run.smithi204.stdout:Building dependency tree... 2020-04-10T06:56:00.438 INFO:teuthology.orchestra.run.smithi204.stdout:Reading state information... 2020-04-10T06:56:00.518 INFO:teuthology.orchestra.run.smithi204.stderr:E: Unable to locate package linux-image-current-generic 2020-04-10T06:56:00.519 DEBUG:teuthology.orchestra.run:got remote process result: 100 2020-04-10T06:56:00.519 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T06:56:00.525 INFO:teuthology.orchestra.run.smithi204:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2020-04-10T06:56:00.650 INFO:teuthology.orchestra.run.smithi204.stdout:Reading package lists... 2020-04-10T06:56:00.888 INFO:teuthology.orchestra.run.smithi204.stdout:Building dependency tree... 2020-04-10T06:56:00.889 INFO:teuthology.orchestra.run.smithi204.stdout:Reading state information... 2020-04-10T06:56:01.048 INFO:teuthology.orchestra.run.smithi204.stdout:The following additional packages will be installed: 2020-04-10T06:56:01.048 INFO:teuthology.orchestra.run.smithi204.stdout: linux-generic linux-headers-4.15.0-96 linux-headers-4.15.0-96-generic 2020-04-10T06:56:01.048 INFO:teuthology.orchestra.run.smithi204.stdout: linux-headers-generic linux-image-4.15.0-96-generic 2020-04-10T06:56:01.048 INFO:teuthology.orchestra.run.smithi204.stdout: linux-modules-4.15.0-96-generic linux-modules-extra-4.15.0-96-generic 2020-04-10T06:56:01.049 INFO:teuthology.orchestra.run.smithi204.stdout:Suggested packages: 2020-04-10T06:56:01.050 INFO:teuthology.orchestra.run.smithi204.stdout: fdutils linux-doc-4.15.0 | linux-source-4.15.0 linux-tools 2020-04-10T06:56:01.050 INFO:teuthology.orchestra.run.smithi204.stdout:Recommended packages: 2020-04-10T06:56:01.050 INFO:teuthology.orchestra.run.smithi204.stdout: thermald 2020-04-10T06:56:01.082 INFO:teuthology.orchestra.run.smithi204.stdout:The following NEW packages will be installed: 2020-04-10T06:56:01.082 INFO:teuthology.orchestra.run.smithi204.stdout: linux-headers-4.15.0-96 linux-headers-4.15.0-96-generic 2020-04-10T06:56:01.082 INFO:teuthology.orchestra.run.smithi204.stdout: linux-image-4.15.0-96-generic linux-modules-4.15.0-96-generic 2020-04-10T06:56:01.082 INFO:teuthology.orchestra.run.smithi204.stdout: linux-modules-extra-4.15.0-96-generic 2020-04-10T06:56:01.084 INFO:teuthology.orchestra.run.smithi204.stdout:The following packages will be upgraded: 2020-04-10T06:56:01.094 INFO:teuthology.orchestra.run.smithi204.stdout: linux-generic linux-headers-generic linux-image-generic 2020-04-10T06:56:01.263 INFO:teuthology.orchestra.run.smithi204.stdout:3 upgraded, 5 newly installed, 0 to remove and 50 not upgraded. 2020-04-10T06:56:01.263 INFO:teuthology.orchestra.run.smithi204.stdout:Need to get 65.9 MB of archives. 2020-04-10T06:56:01.263 INFO:teuthology.orchestra.run.smithi204.stdout:After this operation, 332 MB of additional disk space will be used. 2020-04-10T06:56:01.263 INFO:teuthology.orchestra.run.smithi204.stdout:Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-modules-4.15.0-96-generic amd64 4.15.0-96.97 [13.1 MB] 2020-04-10T06:56:02.284 INFO:teuthology.orchestra.run.smithi204.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-image-4.15.0-96-generic amd64 4.15.0-96.97 [7,998 kB] 2020-04-10T06:56:02.481 INFO:teuthology.orchestra.run.smithi204.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-modules-extra-4.15.0-96-generic amd64 4.15.0-96.97 [32.8 MB] 2020-04-10T06:56:03.455 INFO:teuthology.orchestra.run.smithi204.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-generic amd64 4.15.0.96.87 [1,868 B] 2020-04-10T06:56:03.455 INFO:teuthology.orchestra.run.smithi204.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-image-generic amd64 4.15.0.96.87 [2,468 B] 2020-04-10T06:56:03.461 INFO:teuthology.orchestra.run.smithi204.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-4.15.0-96 all 4.15.0-96.97 [11.0 MB] 2020-04-10T06:56:03.752 INFO:teuthology.orchestra.run.smithi204.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-4.15.0-96-generic amd64 4.15.0-96.97 [1,127 kB] 2020-04-10T06:56:03.778 INFO:teuthology.orchestra.run.smithi204.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-generic amd64 4.15.0.96.87 [2,432 B] 2020-04-10T06:56:04.256 INFO:teuthology.orchestra.run.smithi204.stdout:Fetched 65.9 MB in 3s (24.6 MB/s) 2020-04-10T06:56:04.393 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package linux-modules-4.15.0-96-generic. 2020-04-10T06:56:04.429 INFO:teuthology.orchestra.run.smithi204.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 101956 files and directories currently installed.) 2020-04-10T06:56:04.432 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../0-linux-modules-4.15.0-96-generic_4.15.0-96.97_amd64.deb ... 2020-04-10T06:56:04.481 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking linux-modules-4.15.0-96-generic (4.15.0-96.97) ... 2020-04-10T06:56:06.090 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package linux-image-4.15.0-96-generic. 2020-04-10T06:56:06.104 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../1-linux-image-4.15.0-96-generic_4.15.0-96.97_amd64.deb ... 2020-04-10T06:56:06.213 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking linux-image-4.15.0-96-generic (4.15.0-96.97) ... 2020-04-10T06:56:06.694 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package linux-modules-extra-4.15.0-96-generic. 2020-04-10T06:56:06.706 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../2-linux-modules-extra-4.15.0-96-generic_4.15.0-96.97_amd64.deb ... 2020-04-10T06:56:06.735 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking linux-modules-extra-4.15.0-96-generic (4.15.0-96.97) ... 2020-04-10T06:56:10.378 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../3-linux-generic_4.15.0.96.87_amd64.deb ... 2020-04-10T06:56:10.480 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking linux-generic (4.15.0.96.87) over (4.15.0.91.83) ... 2020-04-10T06:56:10.806 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../4-linux-image-generic_4.15.0.96.87_amd64.deb ... 2020-04-10T06:56:10.934 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking linux-image-generic (4.15.0.96.87) over (4.15.0.91.83) ... 2020-04-10T06:56:11.262 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package linux-headers-4.15.0-96. 2020-04-10T06:56:11.274 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../5-linux-headers-4.15.0-96_4.15.0-96.97_all.deb ... 2020-04-10T06:56:11.311 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking linux-headers-4.15.0-96 (4.15.0-96.97) ... 2020-04-10T06:56:16.057 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package linux-headers-4.15.0-96-generic. 2020-04-10T06:56:16.080 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../6-linux-headers-4.15.0-96-generic_4.15.0-96.97_amd64.deb ... 2020-04-10T06:56:16.114 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking linux-headers-4.15.0-96-generic (4.15.0-96.97) ... 2020-04-10T06:56:18.358 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../7-linux-headers-generic_4.15.0.96.87_amd64.deb ... 2020-04-10T06:56:18.465 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking linux-headers-generic (4.15.0.96.87) over (4.15.0.91.83) ... 2020-04-10T06:56:18.790 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up linux-modules-4.15.0-96-generic (4.15.0-96.97) ... 2020-04-10T06:56:19.338 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up linux-headers-4.15.0-96 (4.15.0-96.97) ... 2020-04-10T06:56:19.439 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up linux-image-4.15.0-96-generic (4.15.0-96.97) ... 2020-04-10T06:56:19.867 INFO:teuthology.orchestra.run.smithi204.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-4.15.0-96-generic 2020-04-10T06:56:19.867 INFO:teuthology.orchestra.run.smithi204.stdout:I: /initrd.img is now a symlink to boot/initrd.img-4.15.0-96-generic 2020-04-10T06:56:20.008 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up linux-modules-extra-4.15.0-96-generic (4.15.0-96.97) ... 2020-04-10T06:56:20.485 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up linux-headers-4.15.0-96-generic (4.15.0-96.97) ... 2020-04-10T06:56:20.611 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up linux-image-generic (4.15.0.96.87) ... 2020-04-10T06:56:20.737 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up linux-headers-generic (4.15.0.96.87) ... 2020-04-10T06:56:20.863 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up linux-generic (4.15.0.96.87) ... 2020-04-10T06:56:20.973 INFO:teuthology.orchestra.run.smithi204.stdout:Processing triggers for linux-image-4.15.0-96-generic (4.15.0-96.97) ... 2020-04-10T06:56:21.168 INFO:teuthology.orchestra.run.smithi204.stdout:/etc/kernel/postinst.d/initramfs-tools: 2020-04-10T06:56:21.168 INFO:teuthology.orchestra.run.smithi204.stdout:update-initramfs: Generating /boot/initrd.img-4.15.0-96-generic 2020-04-10T06:56:36.883 INFO:teuthology.orchestra.run.smithi204.stdout:/etc/kernel/postinst.d/zz-update-grub: 2020-04-10T06:56:36.883 INFO:teuthology.orchestra.run.smithi204.stdout:Sourcing file `/etc/default/grub' 2020-04-10T06:56:36.913 INFO:teuthology.orchestra.run.smithi204.stdout:Generating grub configuration file ... 2020-04-10T06:56:37.275 INFO:teuthology.orchestra.run.smithi204.stdout:Found linux image: /boot/vmlinuz-4.15.0-96-generic 2020-04-10T06:56:37.332 INFO:teuthology.orchestra.run.smithi204.stdout:Found initrd image: /boot/initrd.img-4.15.0-96-generic 2020-04-10T06:56:37.647 INFO:teuthology.orchestra.run.smithi204.stdout:Found linux image: /boot/vmlinuz-4.15.0-91-generic 2020-04-10T06:56:37.677 INFO:teuthology.orchestra.run.smithi204.stdout:Found initrd image: /boot/initrd.img-4.15.0-91-generic 2020-04-10T06:56:39.225 INFO:teuthology.orchestra.run.smithi204.stdout:done 2020-04-10T06:56:39.609 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T06:56:39.615 INFO:teuthology.orchestra.run.smithi204:> dpkg -s linux-image-generic 2020-04-10T06:56:39.683 INFO:teuthology.orchestra.run.smithi204.stdout:Package: linux-image-generic 2020-04-10T06:56:39.683 INFO:teuthology.orchestra.run.smithi204.stdout:Status: install ok installed 2020-04-10T06:56:39.683 INFO:teuthology.orchestra.run.smithi204.stdout:Priority: optional 2020-04-10T06:56:39.683 INFO:teuthology.orchestra.run.smithi204.stdout:Section: kernel 2020-04-10T06:56:39.684 INFO:teuthology.orchestra.run.smithi204.stdout:Installed-Size: 16 2020-04-10T06:56:39.684 INFO:teuthology.orchestra.run.smithi204.stdout:Maintainer: Ubuntu Kernel Team 2020-04-10T06:56:39.684 INFO:teuthology.orchestra.run.smithi204.stdout:Architecture: amd64 2020-04-10T06:56:39.684 INFO:teuthology.orchestra.run.smithi204.stdout:Source: linux-meta 2020-04-10T06:56:39.684 INFO:teuthology.orchestra.run.smithi204.stdout:Version: 4.15.0.96.87 2020-04-10T06:56:39.684 INFO:teuthology.orchestra.run.smithi204.stdout:Depends: linux-image-4.15.0-96-generic, linux-modules-extra-4.15.0-96-generic, linux-firmware, intel-microcode, amd64-microcode 2020-04-10T06:56:39.685 INFO:teuthology.orchestra.run.smithi204.stdout:Recommends: thermald 2020-04-10T06:56:39.685 INFO:teuthology.orchestra.run.smithi204.stdout:Description: Generic Linux kernel image 2020-04-10T06:56:39.685 INFO:teuthology.orchestra.run.smithi204.stdout: This package will always depend on the latest generic kernel image 2020-04-10T06:56:39.685 INFO:teuthology.orchestra.run.smithi204.stdout: available. 2020-04-10T06:56:39.685 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.15.0-91-generic Expected: 4.15.0-96-generic 2020-04-10T06:56:39.686 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2020-04-10T06:56:39.686 DEBUG:teuthology.task.kernel:src is distro, skipping download 2020-04-10T06:56:39.686 DEBUG:teuthology.task.kernel:src is distro, skipping download 2020-04-10T06:56:39.686 INFO:teuthology.task.kernel:Installing distro kernel on cluster1.mon.a... 2020-04-10T06:56:39.686 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi101.front.sepia.ceph.com, path=None, version=distro) 2020-04-10T06:56:39.686 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T06:56:39.692 INFO:teuthology.orchestra.run.smithi101:> sudo apt-get clean 2020-04-10T06:56:39.876 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T06:56:39.885 INFO:teuthology.orchestra.run.smithi101:> sudo apt-get update 2020-04-10T06:56:40.179 INFO:teuthology.orchestra.run.smithi101.stdout:Hit:1 http://archive.ubuntu.com/ubuntu bionic InRelease 2020-04-10T06:56:40.179 INFO:teuthology.orchestra.run.smithi101.stdout:Hit:2 http://security.ubuntu.com/ubuntu bionic-security InRelease 2020-04-10T06:56:40.251 INFO:teuthology.orchestra.run.smithi101.stdout:Hit:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease 2020-04-10T06:56:40.330 INFO:teuthology.orchestra.run.smithi101.stdout:Hit:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease 2020-04-10T06:56:42.059 INFO:teuthology.orchestra.run.smithi101.stdout:Reading package lists... 2020-04-10T06:56:42.104 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T06:56:42.152 INFO:teuthology.orchestra.run.smithi101:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2020-04-10T06:56:42.280 INFO:teuthology.orchestra.run.smithi101.stdout:Reading package lists... 2020-04-10T06:56:42.509 INFO:teuthology.orchestra.run.smithi101.stdout:Building dependency tree... 2020-04-10T06:56:42.509 INFO:teuthology.orchestra.run.smithi101.stdout:Reading state information... 2020-04-10T06:56:42.585 INFO:teuthology.orchestra.run.smithi101.stderr:E: Unable to locate package linux-image-current-generic 2020-04-10T06:56:42.585 DEBUG:teuthology.orchestra.run:got remote process result: 100 2020-04-10T06:56:42.586 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T06:56:42.591 INFO:teuthology.orchestra.run.smithi101:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2020-04-10T06:56:42.720 INFO:teuthology.orchestra.run.smithi101.stdout:Reading package lists... 2020-04-10T06:56:42.974 INFO:teuthology.orchestra.run.smithi101.stdout:Building dependency tree... 2020-04-10T06:56:42.974 INFO:teuthology.orchestra.run.smithi101.stdout:Reading state information... 2020-04-10T06:56:43.155 INFO:teuthology.orchestra.run.smithi101.stdout:linux-image-generic is already the newest version (4.15.0.96.87). 2020-04-10T06:56:43.156 INFO:teuthology.orchestra.run.smithi101.stdout:0 upgraded, 0 newly installed, 0 to remove and 50 not upgraded. 2020-04-10T06:56:43.157 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T06:56:43.162 INFO:teuthology.orchestra.run.smithi101:> dpkg -s linux-image-generic 2020-04-10T06:56:43.219 INFO:teuthology.orchestra.run.smithi101.stdout:Package: linux-image-generic 2020-04-10T06:56:43.219 INFO:teuthology.orchestra.run.smithi101.stdout:Status: install ok installed 2020-04-10T06:56:43.219 INFO:teuthology.orchestra.run.smithi101.stdout:Priority: optional 2020-04-10T06:56:43.220 INFO:teuthology.orchestra.run.smithi101.stdout:Section: kernel 2020-04-10T06:56:43.220 INFO:teuthology.orchestra.run.smithi101.stdout:Installed-Size: 16 2020-04-10T06:56:43.220 INFO:teuthology.orchestra.run.smithi101.stdout:Maintainer: Ubuntu Kernel Team 2020-04-10T06:56:43.220 INFO:teuthology.orchestra.run.smithi101.stdout:Architecture: amd64 2020-04-10T06:56:43.220 INFO:teuthology.orchestra.run.smithi101.stdout:Source: linux-meta 2020-04-10T06:56:43.220 INFO:teuthology.orchestra.run.smithi101.stdout:Version: 4.15.0.96.87 2020-04-10T06:56:43.221 INFO:teuthology.orchestra.run.smithi101.stdout:Depends: linux-image-4.15.0-96-generic, linux-modules-extra-4.15.0-96-generic, linux-firmware, intel-microcode, amd64-microcode 2020-04-10T06:56:43.221 INFO:teuthology.orchestra.run.smithi101.stdout:Recommends: thermald 2020-04-10T06:56:43.221 INFO:teuthology.orchestra.run.smithi101.stdout:Description: Generic Linux kernel image 2020-04-10T06:56:43.221 INFO:teuthology.orchestra.run.smithi101.stdout: This package will always depend on the latest generic kernel image 2020-04-10T06:56:43.221 INFO:teuthology.orchestra.run.smithi101.stdout: available. 2020-04-10T06:56:43.222 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T06:56:43.266 INFO:teuthology.orchestra.run.smithi101:> mktemp 2020-04-10T06:56:43.315 INFO:teuthology.orchestra.run.smithi101.stdout:/tmp/tmp.l0SMCtBsmf 2020-04-10T06:56:43.315 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T06:56:43.362 INFO:teuthology.orchestra.run.smithi101:> sudo cp /boot/grub/grub.cfg /tmp/tmp.l0SMCtBsmf 2020-04-10T06:56:43.420 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T06:56:43.468 INFO:teuthology.orchestra.run.smithi101:> sudo chmod 0666 /tmp/tmp.l0SMCtBsmf 2020-04-10T06:56:43.596 DEBUG:teuthology.orchestra.remote:smithi101:/tmp/tmp.l0SMCtBsmf is 8KB 2020-04-10T06:56:43.611 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T06:56:43.616 INFO:teuthology.orchestra.run.smithi101:> rm -fr /tmp/tmp.l0SMCtBsmf 2020-04-10T06:56:43.668 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T06:56:43.718 INFO:teuthology.orchestra.run.smithi101:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2020-04-10T06:56:43.777 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T06:56:43.823 INFO:teuthology.orchestra.run.smithi101:> sudo sh -c 'cat > /etc/grub.d/01_ceph_kernel' && sudo chmod 755 /etc/grub.d/01_ceph_kernel 2020-04-10T06:56:43.924 INFO:teuthology.task.kernel:Distro Kernel Version: 4.15.0-96-generic 2020-04-10T06:56:43.925 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T06:56:43.973 INFO:teuthology.orchestra.run.smithi101:> sudo update-grub 2020-04-10T06:56:45.206 INFO:teuthology.orchestra.run.smithi101.stderr:Sourcing file `/etc/default/grub' 2020-04-10T06:56:45.228 INFO:teuthology.orchestra.run.smithi101.stderr:Generating grub configuration file ... 2020-04-10T06:56:45.452 INFO:teuthology.orchestra.run.smithi101.stderr:Found linux image: /boot/vmlinuz-4.15.0-96-generic 2020-04-10T06:56:45.465 INFO:teuthology.orchestra.run.smithi101.stderr:Found initrd image: /boot/initrd.img-4.15.0-96-generic 2020-04-10T06:56:45.760 INFO:teuthology.orchestra.run.smithi101.stderr:Found linux image: /boot/vmlinuz-4.15.0-91-generic 2020-04-10T06:56:45.770 INFO:teuthology.orchestra.run.smithi101.stderr:Found initrd image: /boot/initrd.img-4.15.0-91-generic 2020-04-10T06:56:46.960 INFO:teuthology.orchestra.run.smithi101.stderr:done 2020-04-10T06:56:46.960 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T06:56:46.967 INFO:teuthology.orchestra.run.smithi101:> sudo shutdown -r now 2020-04-10T06:56:47.013 INFO:teuthology.task.kernel:Installing distro kernel on cluster2.mon.a... 2020-04-10T06:56:47.013 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi204.front.sepia.ceph.com, path=None, version=distro) 2020-04-10T06:56:47.014 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T06:56:47.020 INFO:teuthology.orchestra.run.smithi204:> sudo apt-get clean 2020-04-10T06:56:47.160 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T06:56:47.165 INFO:teuthology.orchestra.run.smithi204:> sudo apt-get update 2020-04-10T06:56:47.342 INFO:teuthology.orchestra.run.smithi204.stdout:Hit:1 http://security.ubuntu.com/ubuntu bionic-security InRelease 2020-04-10T06:56:47.450 INFO:teuthology.orchestra.run.smithi204.stdout:Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease 2020-04-10T06:56:47.533 INFO:teuthology.orchestra.run.smithi204.stdout:Hit:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease 2020-04-10T06:56:47.611 INFO:teuthology.orchestra.run.smithi204.stdout:Hit:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease 2020-04-10T06:56:49.121 INFO:teuthology.orchestra.run.smithi204.stdout:Reading package lists... 2020-04-10T06:56:49.169 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T06:56:49.175 INFO:teuthology.orchestra.run.smithi204:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2020-04-10T06:56:49.300 INFO:teuthology.orchestra.run.smithi204.stdout:Reading package lists... 2020-04-10T06:56:49.553 INFO:teuthology.orchestra.run.smithi204.stdout:Building dependency tree... 2020-04-10T06:56:49.556 INFO:teuthology.orchestra.run.smithi204.stdout:Reading state information... 2020-04-10T06:56:49.627 INFO:teuthology.orchestra.run.smithi204.stderr:E: Unable to locate package linux-image-current-generic 2020-04-10T06:56:49.629 DEBUG:teuthology.orchestra.run:got remote process result: 100 2020-04-10T06:56:49.629 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T06:56:49.633 INFO:teuthology.orchestra.run.smithi204:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2020-04-10T06:56:49.748 INFO:teuthology.orchestra.run.smithi204.stdout:Reading package lists... 2020-04-10T06:56:49.973 INFO:teuthology.orchestra.run.smithi204.stdout:Building dependency tree... 2020-04-10T06:56:49.973 INFO:teuthology.orchestra.run.smithi204.stdout:Reading state information... 2020-04-10T06:56:50.161 INFO:teuthology.orchestra.run.smithi204.stdout:linux-image-generic is already the newest version (4.15.0.96.87). 2020-04-10T06:56:50.162 INFO:teuthology.orchestra.run.smithi204.stdout:0 upgraded, 0 newly installed, 0 to remove and 50 not upgraded. 2020-04-10T06:56:50.162 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T06:56:50.169 INFO:teuthology.orchestra.run.smithi204:> dpkg -s linux-image-generic 2020-04-10T06:56:50.235 INFO:teuthology.orchestra.run.smithi204.stdout:Package: linux-image-generic 2020-04-10T06:56:50.235 INFO:teuthology.orchestra.run.smithi204.stdout:Status: install ok installed 2020-04-10T06:56:50.235 INFO:teuthology.orchestra.run.smithi204.stdout:Priority: optional 2020-04-10T06:56:50.235 INFO:teuthology.orchestra.run.smithi204.stdout:Section: kernel 2020-04-10T06:56:50.236 INFO:teuthology.orchestra.run.smithi204.stdout:Installed-Size: 16 2020-04-10T06:56:50.236 INFO:teuthology.orchestra.run.smithi204.stdout:Maintainer: Ubuntu Kernel Team 2020-04-10T06:56:50.236 INFO:teuthology.orchestra.run.smithi204.stdout:Architecture: amd64 2020-04-10T06:56:50.239 INFO:teuthology.orchestra.run.smithi204.stdout:Source: linux-meta 2020-04-10T06:56:50.239 INFO:teuthology.orchestra.run.smithi204.stdout:Version: 4.15.0.96.87 2020-04-10T06:56:50.239 INFO:teuthology.orchestra.run.smithi204.stdout:Depends: linux-image-4.15.0-96-generic, linux-modules-extra-4.15.0-96-generic, linux-firmware, intel-microcode, amd64-microcode 2020-04-10T06:56:50.239 INFO:teuthology.orchestra.run.smithi204.stdout:Recommends: thermald 2020-04-10T06:56:50.240 INFO:teuthology.orchestra.run.smithi204.stdout:Description: Generic Linux kernel image 2020-04-10T06:56:50.240 INFO:teuthology.orchestra.run.smithi204.stdout: This package will always depend on the latest generic kernel image 2020-04-10T06:56:50.240 INFO:teuthology.orchestra.run.smithi204.stdout: available. 2020-04-10T06:56:50.240 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T06:56:50.281 INFO:teuthology.orchestra.run.smithi204:> mktemp 2020-04-10T06:56:50.330 INFO:teuthology.orchestra.run.smithi204.stdout:/tmp/tmp.imO02fdmv0 2020-04-10T06:56:50.331 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T06:56:50.382 INFO:teuthology.orchestra.run.smithi204:> sudo cp /boot/grub/grub.cfg /tmp/tmp.imO02fdmv0 2020-04-10T06:56:50.445 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T06:56:50.495 INFO:teuthology.orchestra.run.smithi204:> sudo chmod 0666 /tmp/tmp.imO02fdmv0 2020-04-10T06:56:50.610 DEBUG:teuthology.orchestra.remote:smithi204:/tmp/tmp.imO02fdmv0 is 8KB 2020-04-10T06:56:50.622 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T06:56:50.628 INFO:teuthology.orchestra.run.smithi204:> rm -fr /tmp/tmp.imO02fdmv0 2020-04-10T06:56:50.675 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T06:56:50.721 INFO:teuthology.orchestra.run.smithi204:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2020-04-10T06:56:50.779 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T06:56:50.826 INFO:teuthology.orchestra.run.smithi204:> sudo sh -c 'cat > /etc/grub.d/01_ceph_kernel' && sudo chmod 755 /etc/grub.d/01_ceph_kernel 2020-04-10T06:56:50.924 INFO:teuthology.task.kernel:Distro Kernel Version: 4.15.0-96-generic 2020-04-10T06:56:50.924 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T06:56:50.969 INFO:teuthology.orchestra.run.smithi204:> sudo update-grub 2020-04-10T06:56:52.326 INFO:teuthology.orchestra.run.smithi204.stderr:Sourcing file `/etc/default/grub' 2020-04-10T06:56:52.345 INFO:teuthology.orchestra.run.smithi204.stderr:Generating grub configuration file ... 2020-04-10T06:56:52.601 INFO:teuthology.orchestra.run.smithi204.stderr:Found linux image: /boot/vmlinuz-4.15.0-96-generic 2020-04-10T06:56:52.612 INFO:teuthology.orchestra.run.smithi204.stderr:Found initrd image: /boot/initrd.img-4.15.0-96-generic 2020-04-10T06:56:52.906 INFO:teuthology.orchestra.run.smithi204.stderr:Found linux image: /boot/vmlinuz-4.15.0-91-generic 2020-04-10T06:56:52.918 INFO:teuthology.orchestra.run.smithi204.stderr:Found initrd image: /boot/initrd.img-4.15.0-91-generic 2020-04-10T06:56:54.075 INFO:teuthology.orchestra.run.smithi204.stderr:done 2020-04-10T06:56:54.077 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T06:56:54.082 INFO:teuthology.orchestra.run.smithi204:> sudo shutdown -r now 2020-04-10T06:57:24.157 INFO:teuthology.misc:Re-opening connections... 2020-04-10T06:57:24.157 INFO:teuthology.misc:trying to connect to ubuntu@smithi101.front.sepia.ceph.com 2020-04-10T06:57:24.157 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-04-10T06:57:24.158 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi101.front.sepia.ceph.com', 'timeout': 60} 2020-04-10T06:57:42.613 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.101 2020-04-10T06:57:42.613 INFO:teuthology.misc:trying to connect to ubuntu@smithi204.front.sepia.ceph.com 2020-04-10T06:57:42.613 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-04-10T06:57:42.614 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi204.front.sepia.ceph.com', 'timeout': 60} 2020-04-10T06:58:01.045 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.204 2020-04-10T06:58:01.045 DEBUG:teuthology.misc:waited 36.8881940842 2020-04-10T06:58:02.046 INFO:teuthology.misc:trying to connect to ubuntu@smithi101.front.sepia.ceph.com 2020-04-10T06:58:02.046 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-04-10T06:58:02.047 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi101.front.sepia.ceph.com', 'timeout': 60} 2020-04-10T06:58:33.561 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.101 2020-04-10T06:58:33.561 INFO:teuthology.misc:trying to connect to ubuntu@smithi204.front.sepia.ceph.com 2020-04-10T06:58:33.561 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-04-10T06:58:33.562 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi204.front.sepia.ceph.com', 'timeout': 60} 2020-04-10T06:58:34.581 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.204 2020-04-10T06:58:34.581 DEBUG:teuthology.misc:waited 70.424421072 2020-04-10T06:58:35.582 INFO:teuthology.misc:trying to connect to ubuntu@smithi101.front.sepia.ceph.com 2020-04-10T06:58:35.582 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-04-10T06:58:35.582 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi101.front.sepia.ceph.com', 'timeout': 60} 2020-04-10T06:58:35.583 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.101 2020-04-10T06:58:35.583 INFO:teuthology.misc:trying to connect to ubuntu@smithi204.front.sepia.ceph.com 2020-04-10T06:58:35.583 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-04-10T06:58:35.584 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi204.front.sepia.ceph.com', 'timeout': 60} 2020-04-10T06:58:35.585 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.204 2020-04-10T06:58:35.585 DEBUG:teuthology.misc:waited 71.4286260605 2020-04-10T06:58:36.586 INFO:teuthology.misc:trying to connect to ubuntu@smithi101.front.sepia.ceph.com 2020-04-10T06:58:36.586 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-04-10T06:58:36.586 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi101.front.sepia.ceph.com', 'timeout': 60} 2020-04-10T06:58:36.588 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.101 2020-04-10T06:58:36.588 INFO:teuthology.misc:trying to connect to ubuntu@smithi204.front.sepia.ceph.com 2020-04-10T06:58:36.588 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-04-10T06:58:36.588 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi204.front.sepia.ceph.com', 'timeout': 60} 2020-04-10T06:58:36.589 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.204 2020-04-10T06:58:36.589 DEBUG:teuthology.misc:waited 72.4326341152 2020-04-10T06:58:37.590 INFO:teuthology.misc:trying to connect to ubuntu@smithi101.front.sepia.ceph.com 2020-04-10T06:58:37.590 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-04-10T06:58:37.590 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi101.front.sepia.ceph.com', 'timeout': 60} 2020-04-10T06:58:37.593 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.101 2020-04-10T06:58:37.593 INFO:teuthology.misc:trying to connect to ubuntu@smithi204.front.sepia.ceph.com 2020-04-10T06:58:37.593 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-04-10T06:58:37.595 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi204.front.sepia.ceph.com', 'timeout': 60} 2020-04-10T06:58:37.603 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.204 2020-04-10T06:58:37.603 DEBUG:teuthology.misc:waited 73.44632411 2020-04-10T06:58:38.603 INFO:teuthology.misc:trying to connect to ubuntu@smithi101.front.sepia.ceph.com 2020-04-10T06:58:38.604 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-04-10T06:58:38.604 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi101.front.sepia.ceph.com', 'timeout': 60} 2020-04-10T06:58:38.815 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T06:58:39.816 DEBUG:teuthology.misc:waited 75.6597061157 2020-04-10T06:58:40.818 INFO:teuthology.misc:trying to connect to ubuntu@smithi204.front.sepia.ceph.com 2020-04-10T06:58:40.818 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-04-10T06:58:40.819 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi204.front.sepia.ceph.com', 'timeout': 60} 2020-04-10T06:58:40.893 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T06:58:41.208 DEBUG:teuthology.misc:waited 77.0509450436 2020-04-10T06:58:42.209 INFO:teuthology.task.kernel:Checking client cluster1.mon.a for new kernel version... 2020-04-10T06:58:42.209 INFO:teuthology.task.kernel:Checking kernel version of cluster1.mon.a, want "4.15.0-96-generic"... 2020-04-10T06:58:42.209 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T06:58:42.215 INFO:teuthology.orchestra.run.smithi101:> uname -r 2020-04-10T06:58:42.266 INFO:teuthology.orchestra.run.smithi101.stdout:4.15.0-96-generic 2020-04-10T06:58:42.266 DEBUG:teuthology.task.kernel:current kernel version is 4.15.0-96-generic vs 4.15.0-96-generic 2020-04-10T06:58:42.266 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2020-04-10T06:58:42.266 INFO:teuthology.task.kernel:Checking client cluster2.mon.a for new kernel version... 2020-04-10T06:58:42.266 INFO:teuthology.task.kernel:Checking kernel version of cluster2.mon.a, want "4.15.0-96-generic"... 2020-04-10T06:58:42.267 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T06:58:42.272 INFO:teuthology.orchestra.run.smithi204:> uname -r 2020-04-10T06:58:42.321 INFO:teuthology.orchestra.run.smithi204.stdout:4.15.0-96-generic 2020-04-10T06:58:42.322 DEBUG:teuthology.task.kernel:current kernel version is 4.15.0-96-generic vs 4.15.0-96-generic 2020-04-10T06:58:42.322 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2020-04-10T06:58:43.325 INFO:teuthology.task.kernel:Enabling kdb on cluster1.mon.a... 2020-04-10T06:58:43.325 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T06:58:43.335 INFO:teuthology.orchestra.run.smithi101:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2020-04-10T06:58:43.510 INFO:teuthology.orchestra.run.smithi101.stdout:ttyS1 2020-04-10T06:58:43.529 INFO:teuthology.task.kernel:Enabling kdb on cluster2.mon.a... 2020-04-10T06:58:43.529 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T06:58:43.534 INFO:teuthology.orchestra.run.smithi204:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2020-04-10T06:58:43.670 INFO:teuthology.orchestra.run.smithi204.stdout:ttyS1 2020-04-10T06:58:43.693 INFO:teuthology.run_tasks:Running task internal.base... 2020-04-10T06:58:43.706 INFO:teuthology.task.internal:Creating test directory... 2020-04-10T06:58:43.706 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T06:58:43.712 INFO:teuthology.orchestra.run.smithi101:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2020-04-10T06:58:43.756 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T06:58:43.761 INFO:teuthology.orchestra.run.smithi204:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2020-04-10T06:58:43.808 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2020-04-10T06:58:43.818 INFO:teuthology.run_tasks:Running task internal.archive... 2020-04-10T06:58:43.837 INFO:teuthology.task.internal:Creating archive directory... 2020-04-10T06:58:43.837 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T06:58:43.859 INFO:teuthology.orchestra.run.smithi101:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2020-04-10T06:58:43.905 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T06:58:43.910 INFO:teuthology.orchestra.run.smithi204:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2020-04-10T06:58:43.974 INFO:teuthology.run_tasks:Running task internal.coredump... 2020-04-10T06:58:43.989 INFO:teuthology.task.internal:Enabling coredump saving... 2020-04-10T06:58:43.989 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T06:58:43.997 INFO:teuthology.orchestra.run.smithi101:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2020-04-10T06:58:44.042 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T06:58:44.050 INFO:teuthology.orchestra.run.smithi204:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2020-04-10T06:58:44.058 INFO:teuthology.orchestra.run.smithi101.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-04-10T06:58:44.068 INFO:teuthology.orchestra.run.smithi101.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-04-10T06:58:44.114 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-04-10T06:58:44.121 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-04-10T06:58:44.122 INFO:teuthology.run_tasks:Running task internal.sudo... 2020-04-10T06:58:44.138 INFO:teuthology.task.internal:Configuring sudo... 2020-04-10T06:58:44.138 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T06:58:44.144 INFO:teuthology.orchestra.run.smithi101:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2020-04-10T06:58:44.188 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T06:58:44.194 INFO:teuthology.orchestra.run.smithi204:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2020-04-10T06:58:44.250 INFO:teuthology.run_tasks:Running task internal.syslog... 2020-04-10T06:58:44.261 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2020-04-10T06:58:44.261 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T06:58:44.266 INFO:teuthology.orchestra.run.smithi101:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2020-04-10T06:58:44.308 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T06:58:44.314 INFO:teuthology.orchestra.run.smithi204:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2020-04-10T06:58:44.360 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T06:58:44.366 INFO:teuthology.orchestra.run.smithi101:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2020-04-10T06:58:44.413 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T06:58:44.459 INFO:teuthology.orchestra.run.smithi101:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2020-04-10T06:58:44.509 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T06:58:44.555 INFO:teuthology.orchestra.run.smithi101:> sudo sh -c 'cat > /etc/rsyslog.d/80-cephtest.conf' 2020-04-10T06:58:44.645 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T06:58:44.650 INFO:teuthology.orchestra.run.smithi204:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2020-04-10T06:58:44.703 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T06:58:44.751 INFO:teuthology.orchestra.run.smithi204:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2020-04-10T06:58:44.801 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T06:58:44.847 INFO:teuthology.orchestra.run.smithi204:> sudo sh -c 'cat > /etc/rsyslog.d/80-cephtest.conf' 2020-04-10T06:58:44.937 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T06:58:44.942 INFO:teuthology.orchestra.run.smithi101:> sudo service rsyslog restart 2020-04-10T06:58:44.984 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T06:58:44.990 INFO:teuthology.orchestra.run.smithi204:> sudo service rsyslog restart 2020-04-10T06:58:45.076 INFO:teuthology.run_tasks:Running task internal.timer... 2020-04-10T06:58:45.086 INFO:teuthology.task.internal:Starting timer... 2020-04-10T06:58:45.087 INFO:teuthology.run_tasks:Running task pcp... 2020-04-10T06:58:45.125 INFO:teuthology.run_tasks:Running task selinux... 2020-04-10T06:58:45.140 DEBUG:teuthology.task.selinux:Excluding smithi101: OS 'ubuntu' does not support SELinux 2020-04-10T06:58:45.140 DEBUG:teuthology.task.selinux:Excluding smithi204: OS 'ubuntu' does not support SELinux 2020-04-10T06:58:45.140 DEBUG:teuthology.task.selinux:Getting current SELinux state 2020-04-10T06:58:45.141 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2020-04-10T06:58:45.141 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2020-04-10T06:58:45.141 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2020-04-10T06:58:45.154 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master was just updated; assuming it is current 2020-04-10T06:58:45.154 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master to branch origin/master 2020-04-10T06:58:45.171 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}] 2020-04-10T06:58:45.174 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi101.front.sepia.ceph.com,smithi204.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/cephlab.yml 2020-04-10T06:58:45.522 INFO:teuthology.task.ansible.out:Using /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/ansible.cfg as config file 2020-04-10T06:58:46.651 INFO:teuthology.task.ansible.out: [WARNING]: file /home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/roles/testnode/tasks/setup-opensuse.yml is empty and had no tasks to include 2020-04-10T06:58:49.587 INFO:teuthology.task.ansible.out: PLAY [all] ********************************************************************* 2020-04-10T06:58:49.612 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:58:49 +0000 (0:00:00.066) 0:00:00.066 ********** 2020-04-10T06:58:49.624 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:58:49 +0000 (0:00:00.011) 0:00:00.077 ********** 2020-04-10T06:58:52.434 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* 2020-04-10T06:58:52.436 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] 2020-04-10T06:58:52.516 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] 2020-04-10T06:58:52.597 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:58:52 +0000 (0:00:02.972) 0:00:03.050 ********** 2020-04-10T06:58:52.640 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:58:52 +0000 (0:00:00.043) 0:00:03.093 ********** 2020-04-10T06:58:52.879 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the sudo group.] ******************************** 2020-04-10T06:58:52.880 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} ok: [smithi204.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2020-04-10T06:58:52.919 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:58:52 +0000 (0:00:00.278) 0:00:03.372 ********** 2020-04-10T06:58:52.953 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:58:52 +0000 (0:00:00.034) 0:00:03.407 ********** 2020-04-10T06:58:53.545 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the ansible user.] ****************************** 2020-04-10T06:58:53.547 INFO:teuthology.task.ansible.out:changed: [smithi204.front.sepia.ceph.com] => {"append": false, "changed": true, "comment": "cm,,,", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2020-04-10T06:58:53.585 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:58:53 +0000 (0:00:00.631) 0:00:04.038 ********** 2020-04-10T06:58:53.873 INFO:teuthology.task.ansible.out:changed: [smithi101.front.sepia.ceph.com] => {"append": false, "changed": true, "comment": "cm,,,", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2020-04-10T06:58:53.913 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:58:53 +0000 (0:00:00.327) 0:00:04.366 ********** 2020-04-10T06:58:53.928 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Delete the ansible users password.] ******************** 2020-04-10T06:58:53.929 INFO:teuthology.task.ansible.out:changed: [smithi204.front.sepia.ceph.com] => {"changed": true, "cmd": ["passwd", "-d", "cm"], "delta": "0:00:00.071319", "end": "2020-04-10 06:58:54.162005", "rc": 0, "start": "2020-04-10 06:58:54.090686", "stderr": "", "stderr_lines": [], "stdout": "passwd: password expiry information changed.", "stdout_lines": ["passwd: password expiry information changed."]} 2020-04-10T06:58:53.971 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:58:53 +0000 (0:00:00.057) 0:00:04.424 ********** 2020-04-10T06:58:54.112 INFO:teuthology.task.ansible.out:changed: [smithi101.front.sepia.ceph.com] => {"changed": true, "cmd": ["passwd", "-d", "cm"], "delta": "0:00:00.071121", "end": "2020-04-10 06:58:54.093502", "rc": 0, "start": "2020-04-10 06:58:54.022381", "stderr": "", "stderr_lines": [], "stdout": "passwd: password expiry information changed.", "stdout_lines": ["passwd: password expiry information changed."]} 2020-04-10T06:58:54.153 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:58:54 +0000 (0:00:00.181) 0:00:04.606 ********** 2020-04-10T06:58:54.236 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Ensure includedir is present in sudoers.] ************** 2020-04-10T06:58:54.238 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-04-10T06:58:54.291 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:58:54 +0000 (0:00:00.138) 0:00:04.744 ********** 2020-04-10T06:58:54.296 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-04-10T06:58:54.337 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:58:54 +0000 (0:00:00.045) 0:00:04.790 ********** 2020-04-10T06:58:55.118 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the cephlab_sudo sudoers.d file.] *************** 2020-04-10T06:58:55.119 INFO:teuthology.task.ansible.out:changed: [smithi204.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1586501934.31-255215117980880/source", "state": "file", "uid": 0} 2020-04-10T06:58:55.173 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:58:55 +0000 (0:00:00.835) 0:00:05.626 ********** 2020-04-10T06:58:55.181 INFO:teuthology.task.ansible.out:changed: [smithi101.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1586501934.35-274741559459930/source", "state": "file", "uid": 0} 2020-04-10T06:58:55.227 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:58:55 +0000 (0:00:00.054) 0:00:05.680 ********** 2020-04-10T06:58:55.610 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Add authorized keys for the ansible user.] ************* 2020-04-10T06:58:55.611 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv 2020-04-10T06:58:55.612 INFO:teuthology.task.ansible.out:4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} ok: [smithi204.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail 2020-04-10T06:58:55.613 INFO:teuthology.task.ansible.out:.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@ 2020-04-10T06:58:55.615 INFO:teuthology.task.ansible.out:aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2020-04-10T06:58:55.782 INFO:teuthology.task.ansible.out: PLAY [teuthology] ************************************************************** 2020-04-10T06:58:55.784 INFO:teuthology.task.ansible.out:skipping: no hosts matched 2020-04-10T06:58:55.789 INFO:teuthology.task.ansible.out: PLAY [testnodes] *************************************************************** 2020-04-10T06:58:55.980 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:58:55 +0000 (0:00:00.753) 0:00:06.433 ********** 2020-04-10T06:58:56.019 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:58:56 +0000 (0:00:00.039) 0:00:06.472 ********** 2020-04-10T06:58:56.716 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi204.front.sepia.ceph.com] 2020-04-10T06:58:56.778 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] 2020-04-10T06:58:56.855 INFO:teuthology.task.ansible.out: [WARNING]: Using run_once with the free strategy is not currently supported. This task will still be executed for every host in the inventory list. Friday 10 April 2020 06:58:56 +0000 (0:00:00.831) 0:00:07.304 ********** 2020-04-10T06:58:56.900 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:58:56 +0000 (0:00:00.048) 0:00:07.353 ********** 2020-04-10T06:58:57.209 INFO:teuthology.task.ansible.out: TASK [users : Check keys_repo HEAD sha1] *************************************** 2020-04-10T06:58:57.210 INFO:teuthology.task.ansible.out:changed: [smithi204.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.223732", "end": "2020-04-10 06:58:57.187199", "rc": 0, "start": "2020-04-10 06:58:56.963467", "stderr": "", "stderr_lines": [], "stdout": "f5045e16501fc514de59d088aafd14d076981fdf", "stdout_lines": ["f5045e16501fc514de59d088aafd14d076981fdf"]} 2020-04-10T06:58:57.260 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:58:57 +0000 (0:00:00.359) 0:00:07.713 ********** 2020-04-10T06:58:57.266 INFO:teuthology.task.ansible.out:changed: [smithi101.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.220636", "end": "2020-04-10 06:58:57.238885", "rc": 0, "start": "2020-04-10 06:58:57.018249", "stderr": "", "stderr_lines": [], "stdout": "f5045e16501fc514de59d088aafd14d076981fdf", "stdout_lines": ["f5045e16501fc514de59d088aafd14d076981fdf"]} 2020-04-10T06:58:57.309 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:58:57 +0000 (0:00:00.049) 0:00:07.762 ********** 2020-04-10T06:58:57.410 INFO:teuthology.task.ansible.out: TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** changed: [smithi204.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.017713", "end": "2020-04-10 06:58:57.646465", "failed_when_result": false, "rc": 0, "start": "2020-04-10 06:58:57.628752", "stderr": "", "stderr_lines": [], "stdout": "b22ad6dbd27eb2f944823eb8be286527dd352386", "stdout_lines": ["b22ad6dbd27eb2f944823eb8be286527dd352386"]} 2020-04-10T06:58:57.453 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:58:57 +0000 (0:00:00.143) 0:00:07.906 ********** 2020-04-10T06:58:57.455 INFO:teuthology.task.ansible.out: 2020-04-10T06:58:57.462 INFO:teuthology.task.ansible.out:changed: [smithi101.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.005587", "end": "2020-04-10 06:58:57.428372", "failed_when_result": false, "rc": 0, "start": "2020-04-10 06:58:57.422785", "stderr": "", "stderr_lines": [], "stdout": "b22ad6dbd27eb2f944823eb8be286527dd352386", "stdout_lines": ["b22ad6dbd27eb2f944823eb8be286527dd352386"]} 2020-04-10T06:58:57.463 INFO:teuthology.task.ansible.out: 2020-04-10T06:58:57.504 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:58:57 +0000 (0:00:00.050) 0:00:07.957 ********** 2020-04-10T06:58:57.510 INFO:teuthology.task.ansible.out: TASK [users : Determine if we can skip users and pubkeys updates] ************** 2020-04-10T06:58:57.511 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:58:57.554 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:58:57 +0000 (0:00:00.050) 0:00:08.007 ********** 2020-04-10T06:58:57.561 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:58:57.604 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:58:57 +0000 (0:00:00.049) 0:00:08.057 ********** 2020-04-10T06:58:57.611 INFO:teuthology.task.ansible.out: TASK [users : Merge extra_admin_users into managed_admin_users] **************** skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:58:57.653 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:58:57 +0000 (0:00:00.048) 0:00:08.106 ********** 2020-04-10T06:58:57.658 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:58:57.703 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:58:57 +0000 (0:00:00.050) 0:00:08.156 ********** 2020-04-10T06:58:57.708 INFO:teuthology.task.ansible.out: TASK [users : Remove managed_admin_users from managed_users] ******************* 2020-04-10T06:58:57.710 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:58:57.755 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:58:57 +0000 (0:00:00.051) 0:00:08.208 ********** 2020-04-10T06:58:57.761 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:58:57.804 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:58:57 +0000 (0:00:00.048) 0:00:08.257 ********** 2020-04-10T06:58:57.809 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_users list] *********************************** 2020-04-10T06:58:57.810 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:58:57.852 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:58:57 +0000 (0:00:00.046) 0:00:08.303 ********** 2020-04-10T06:58:57.856 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:58:57.898 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:58:57 +0000 (0:00:00.047) 0:00:08.351 ********** 2020-04-10T06:58:57.903 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_admin_users list] ***************************** 2020-04-10T06:58:57.905 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:58:57.944 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:58:57 +0000 (0:00:00.046) 0:00:08.397 ********** 2020-04-10T06:58:57.950 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:58:57.994 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:58:57 +0000 (0:00:00.049) 0:00:08.447 ********** 2020-04-10T06:58:58.069 INFO:teuthology.task.ansible.out: TASK [users : Set UID_MIN to 1001] ********************************************* ok: [smithi204.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-04-10T06:58:58.112 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:58:58 +0000 (0:00:00.117) 0:00:08.565 ********** 2020-04-10T06:58:58.139 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-04-10T06:58:58.188 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:58:58 +0000 (0:00:00.076) 0:00:08.641 ********** 2020-04-10T06:58:58.304 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2020-04-10T06:58:58.305 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9', u'name': u'andrewschoen'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1101, "groups": "sudo", "home": "/home/andrewschoen", "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "move_home": false, "name": "andrewschoen", "shell": "/bin/bash", "state": "present", "uid": 1101} 2020-04-10T06:58:58.381 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9', u'name': u'andrewschoen'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1101, "groups": "sudo", "home": "/home/andrewschoen", "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "move_home": false, "name": "andrewschoen", "shell": "/bin/bash", "state": "present", "uid": 1101} 2020-04-10T06:58:58.446 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n', u'name': u'zack'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1102, "groups": "sudo", "home": "/home/zack", "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "move_home": false, "name": "zack", "shell": "/bin/bash", "state": "present", "uid": 1102} 2020-04-10T06:58:58.513 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n', u'name': u'zack'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1102, "groups": "sudo", "home": "/home/zack", "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "move_home": false, "name": "zack", "shell": "/bin/bash", "state": "present", "uid": 1102} 2020-04-10T06:58:58.596 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd', u'name': u'kdreyer'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1103, "groups": "sudo", "home": "/home/kdreyer", "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "move_home": false, "name": "kdreyer", "shell": "/bin/bash", "state": "present", "uid": 1103} 2020-04-10T06:58:58.651 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd', u'name': u'kdreyer'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1103, "groups": "sudo", "home": "/home/kdreyer", "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "move_home": false, "name": "kdreyer", "shell": "/bin/bash", "state": "present", "uid": 1103} 2020-04-10T06:58:58.727 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n', u'name': u'dmick'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1104, "groups": "sudo", "home": "/home/dmick", "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "move_home": false, "name": "dmick", "shell": "/bin/bash", "state": "present", "uid": 1104} 2020-04-10T06:58:58.790 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n', u'name': u'dmick'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1104, "groups": "sudo", "home": "/home/dmick", "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af598 2020-04-10T06:58:58.792 INFO:teuthology.task.ansible.out:94c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "move_home": false, "name": "dmick", "shell": "/bin/bash", "state": "present", "uid": 1104} 2020-04-10T06:58:58.878 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n', u'name': u'sage'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1105, "groups": "sudo", "home": "/home/sage", "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "move_home": false, "name": "sage", "shell": "/bin/bash", "state": "present", "uid": 1105} 2020-04-10T06:58:58.933 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n', u'name': u'sage'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1105, "groups": "sudo", "home": "/home/sage", "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "move_home": false, "name": "sage", "shell": "/bin/bash", "state": "present", "uid": 1105} 2020-04-10T06:58:59.029 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n', u'name': u'dgalloway'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1106, "groups": "sudo", "home": "/home/dgalloway", "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "move_home": false, "name": "dgalloway", "shell": "/bin/bash", "state": "present", "uid": 1106} 2020-04-10T06:58:59.077 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n', u'name': u'dgalloway'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1106, "groups": "sudo", "home": "/home/dgalloway", "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "move_home": false, "name": "dgalloway", "shell": "/bin/bash", "state": "present", "uid": 1106} 2020-04-10T06:58:59.167 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045', u'name': u'akraitma'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1107, "groups": "sudo", "home": "/home/akraitma", "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "move_home": false, "name": "akraitma", "shell": "/bin/bash", "state": "present", "uid": 1107} 2020-04-10T06:58:59.210 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045', u'name': u'akraitma'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1107, "groups": "sudo", "home": "/home/akraitma", "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "move_home": false, "name": "akraitma", "shell": "/bin/bash", "state": "present", "uid": 1107} 2020-04-10T06:58:59.834 INFO:teuthology.task.ansible.out:changed: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9', u'name': u'jcaratzas'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10103, "groups": "sudo", "home": "/home/jcaratzas", "item": {"name": "jcaratzas", "ovpn": "jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9"}, "name": "jcaratzas", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10102} 2020-04-10T06:58:59.845 INFO:teuthology.task.ansible.out:changed: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9', u'name': u'jcaratzas'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10103, "groups": "sudo", "home": "/home/jcaratzas", "item": {"name": "jcaratzas", "ovpn": "jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9"}, "name": "jcaratzas", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10102} 2020-04-10T06:58:59.976 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e', u'name': u'trociny'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1108, "groups": "sudo", "home": "/home/trociny", "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "move_home": false, "name": "trociny", "shell": "/bin/bash", "state": "present", "uid": 1108} 2020-04-10T06:58:59.987 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e', u'name': u'trociny'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1108, "groups": "sudo", "home": "/home/trociny", "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "move_home": false, "name": "trociny", "shell": "/bin/bash", "state": "present", "uid": 1108} 2020-04-10T06:59:00.125 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007', u'name': u'smithfarm', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1109, "groups": "sudo", "home": "/home/smithfarm", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "move_home": false, "name": "smithfarm", "shell": "/bin/bash", "state": "present", "uid": 1109} 2020-04-10T06:59:00.137 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007', u'name': u'smithfarm', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1109, "groups": "sudo", "home": "/home/smithfarm", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "move_home": false, "name": "smithfarm", "shell": "/bin/bash", "state": "present", "uid": 1109} 2020-04-10T06:59:00.270 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n', u'name': u'mbenjamin'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1110, "groups": "sudo", "home": "/home/mbenjamin", "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "move_home": false, "name": "mbenjamin", "shell": "/bin/bash", "state": "present", "uid": 1110} ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n', u'name': u'mbenjamin'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1110, "groups": "sudo", "home": "/home/mbenjamin", "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\n 2020-04-10T06:59:00.273 INFO:teuthology.task.ansible.out:mbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "move_home": false, "name": "mbenjamin", "shell": "/bin/bash", "state": "present", "uid": 1110} 2020-04-10T06:59:00.416 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01', u'name': u'aemerson', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1111, "groups": "sudo", "home": "/home/aemerson", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "move_home": false, "name": "aemerson", "shell": "/bin/bash", "state": "present", "uid": 1111} 2020-04-10T06:59:00.427 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01', u'name': u'aemerson', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1111, "groups": "sudo", "home": "/home/aemerson", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "move_home": false, "name": "aemerson", "shell": "/bin/bash", "state": "present", "uid": 1111} 2020-04-10T06:59:00.557 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78', u'name': u'amaredia', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1112, "groups": "sudo", "home": "/home/amaredia", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "move_home": false, "name": "amaredia", "shell": "/bin/bash", "state": "present", "uid": 1112} 2020-04-10T06:59:00.569 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78', u'name': u'amaredia', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1112, "groups": "sudo", "home": "/home/amaredia", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "move_home": false, "name": "amaredia", "shell": "/bin/bash", "state": "present", "uid": 1112} 2020-04-10T06:59:00.698 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5', u'name': u'tserlin', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1113, "groups": "sudo", "home": "/home/tserlin", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "move_home": false, "name": "tserlin", "shell": "/bin/bash", "state": "present", "uid": 1113} 2020-04-10T06:59:00.710 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5', u'name': u'tserlin', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1113, "groups": "sudo", "home": "/home/tserlin", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "move_home": false, "name": "tserlin", "shell": "/bin/bash", "state": "present", "uid": 1113} 2020-04-10T06:59:00.831 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28', u'name': u'dis'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1114, "groups": "sudo", "home": "/home/dis", "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "move_home": false, "name": "dis", "shell": "/bin/bash", "state": "present", "uid": 1114} 2020-04-10T06:59:00.843 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28', u'name': u'dis'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1114, "groups": "sudo", "home": "/home/dis", "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "move_home": false, "name": "dis", "shell": "/bin/bash", "state": "present", "uid": 1114} 2020-04-10T06:59:00.959 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n', u'name': u'gregf'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1115, "groups": "sudo", "home": "/home/gregf", "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "move_home": false, "name": "gregf", "shell": "/bin/bash", "state": "present", "uid": 1115} 2020-04-10T06:59:00.981 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n', u'name': u'gregf'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1115, "groups": "sudo", "home": "/home/gregf", "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "move_home": false, "name": "gregf", "shell": "/bin/bash", "state": "present", "uid": 1115} 2020-04-10T06:59:01.090 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n', u'name': u'joshd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1116, "groups": "sudo", "home": "/home/joshd", "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "move_home": false, "name": "joshd", "shell": "/bin/bash", "state": "present", "uid": 1116} 2020-04-10T06:59:01.142 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n', u'name': u'joshd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1116, "groups": "sudo", "home": "/home/joshd", "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "move_home": false, "name": "joshd", "shell": "/bin/bash", "state": "present", "uid": 1116} 2020-04-10T06:59:01.219 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n', u'name': u'davidz'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1117, "groups": "sudo", "home": "/home/davidz", "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "move_home": false, "name": "davidz", "shell": "/bin/bash", "state": "present", "uid": 1117} 2020-04-10T06:59:01.287 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n', u'name': u'davidz'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1117, "groups": "sudo", "home": "/home/davidz", "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "move_home": false, "name": "davidz", "shell": "/bin/bash", "state": "present", "uid": 1117} 2020-04-10T06:59:01.778 INFO:teuthology.task.ansible.out:changed: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed', u'name': u'cmeno'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10104, "groups": "sudo", "home": "/home/cmeno", "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "name": "cmeno", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10103} 2020-04-10T06:59:01.806 INFO:teuthology.task.ansible.out:changed: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed', u'name': u'cmeno'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10104, "groups": "sudo", "home": "/home/cmeno", "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "name": "cmeno", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10103} 2020-04-10T06:59:01.946 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1', u'name': u'ivancich'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1119, "groups": "sudo", "home": "/home/ivancich", "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "move_home": false, "name": "ivancich", "shell": "/bin/bash", "state": "present", "uid": 1119} 2020-04-10T06:59:01.947 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1', u'name': u'ivancich'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1119, "groups": "sudo", "home": "/home/ivancich", "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "move_home": false, "name": "ivancich", "shell": "/bin/bash", "state": "present", "uid": 1119} 2020-04-10T06:59:02.078 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n', u'name': u'wusui'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1120, "groups": "sudo", "home": "/home/wusui", "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "move_home": false, "name": "wusui", "shell": "/bin/bash", "state": "present", "uid": 1120} 2020-04-10T06:59:02.091 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n', u'name': u'wusui'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1120, "groups": "sudo", "home": "/home/wusui", "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "move_home": false, "name": "wusui", "shell": "/bin/bash", "state": "present", "uid": 1120} 2020-04-10T06:59:02.219 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304', u'name': u'zyan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1121, "groups": "sudo", "home": "/home/zyan", "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "move_home": false, "name": "zyan", "shell": "/bin/bash", "state": "present", "uid": 1121} 2020-04-10T06:59:02.245 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304', u'name': u'zyan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1121, "groups": "sudo", "home": "/home/zyan", "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "move_home": false, "name": "zyan", "shell": "/bin/bash", "state": "present", "uid": 1121} 2020-04-10T06:59:02.373 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n', u'name': u'yuriw'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1122, "groups": "sudo", "home": "/home/yuriw", "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "move_home": false, "name": "yuriw", "shell": "/bin/bash", "state": "present", "uid": 1122} 2020-04-10T06:59:02.383 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n', u'name': u'yuriw'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1122, "groups": "sudo", "home": "/home/yuriw", "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "move_home": false, "name": "yuriw", "shell": "/bin/bash", "state": "present", "uid": 1122} 2020-04-10T06:59:02.520 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n', u'name': u'tamil'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1123, "groups": "sudo", "home": "/home/tamil", "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "move_home": false, "name": "tamil", "shell": "/bin/bash", "state": "present", "uid": 1123} 2020-04-10T06:59:02.522 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n', u'name': u'tamil'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1123, "groups": "sudo", "home": "/home/tamil", "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "move_home": false, "name": "tamil", "shell": "/bin/bash", "state": "present", "uid": 1123} 2020-04-10T06:59:02.653 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n', u'name': u'jowilkin'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1124, "groups": "sudo", "home": "/home/jowilkin", "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "move_home": false, "name": "jowilkin", "shell": "/bin/bash", "state": "present", "uid": 1124} 2020-04-10T06:59:02.664 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n', u'name': u'jowilkin'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1124, "groups": "sudo", "home": "/home/jowilkin", "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "move_home": false, "name": "jowilkin", "shell": "/bin/bash", "state": "present", "uid": 1124} 2020-04-10T06:59:02.811 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4', u'name': u'bhubbard'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1125, "groups": "sudo", "home": "/home/bhubbard", "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "move_home": false, "name": "bhubbard", "shell": "/bin/bash", "state": "present", "uid": 1125} 2020-04-10T06:59:02.823 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4', u'name': u'bhubbard'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1125, "groups": "sudo", "home": "/home/bhubbard", "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "move_home": false, "name": "bhubbard", "shell": "/bin/bash", "state": "present", "uid": 1125} 2020-04-10T06:59:02.951 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n', u'name': u'yehudasa'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1126, "groups": "sudo", "home": "/home/yehudasa", "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "move_home": false, "name": "yehudasa", "shell": "/bin/bash", "state": "present", "uid": 1126} 2020-04-10T06:59:02.975 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n', u'name': u'yehudasa'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1126, "groups": "sudo", "home": "/home/yehudasa", "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "move_home": false, "name": "yehudasa", "shell": "/bin/bash", "state": "present", "uid": 1126} 2020-04-10T06:59:03.104 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4', u'name': u'dang'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1127, "groups": "sudo", "home": "/home/dang", "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "move_home": false, "name": "dang", "shell": "/bin/bash", "state": "present", "uid": 1127} 2020-04-10T06:59:03.116 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4', u'name': u'dang'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1127, "groups": "sudo", "home": "/home/dang", "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "move_home": false, "name": "dang", "shell": "/bin/bash", "state": "present", "uid": 1127} 2020-04-10T06:59:03.257 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb', u'name': u'branto'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1128, "groups": "sudo", "home": "/home/branto", "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "move_home": false, "name": "branto", "shell": "/bin/bash", "state": "present", "uid": 1128} 2020-04-10T06:59:03.274 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb', u'name': u'branto'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1128, "groups": "sudo", "home": "/home/branto", "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "move_home": false, "name": "branto", "shell": "/bin/bash", "state": "present", "uid": 1128} 2020-04-10T06:59:03.446 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6', u'name': u'xiaoxichen'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1129, "groups": "sudo", "home": "/home/xiaoxichen", "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "move_home": false, "name": "xiaoxichen", "shell": "/bin/bash", "state": "present", "uid": 1129} ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6', u'name': u'xiaoxichen'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1129, "groups": "sudo", "home": "/home/xiaoxichen", "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "move_home": false, "name": "xiaoxichen", "shell": "/bin/bash", "state": "present", "uid": 1129} 2020-04-10T06:59:03.574 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e', u'name': u'ffilz'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1130, "groups": "sudo", "home": "/home/ffilz", "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "move_home": false, "name": "ffilz", "shell": "/bin/bash", "state": "present", "uid": 1130} 2020-04-10T06:59:03.609 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e', u'name': u'ffilz'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1130, "groups": "sudo", "home": "/home/ffilz", "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "move_home": false, "name": "ffilz", "shell": "/bin/bash", "state": "present", "uid": 1130} 2020-04-10T06:59:03.722 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n', u'name': u'joao'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1131, "groups": "sudo", "home": "/home/joao", "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "move_home": false, "name": "joao", "shell": "/bin/bash", "state": "present", "uid": 1131} 2020-04-10T06:59:03.754 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n', u'name': u'joao'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1131, "groups": "sudo", "home": "/home/joao", "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "move_home": false, "name": "joao", "shell": "/bin/bash", "state": "present", "uid": 1131} 2020-04-10T06:59:03.861 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n', u'name': u'nhm'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1132, "groups": "sudo", "home": "/home/nhm", "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "move_home": false, "name": "nhm", "shell": "/bin/bash", "state": "present", "uid": 1132} 2020-04-10T06:59:03.916 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n', u'name': u'nhm'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1132, "groups": "sudo", "home": "/home/nhm", "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "move_home": false, "name": "nhm", "shell": "/bin/bash", "state": "present", "uid": 1132} 2020-04-10T06:59:04.000 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313', u'name': u'nwatkins'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1133, "groups": "sudo", "home": "/home/nwatkins", "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "move_home": false, "name": "nwatkins", "shell": "/bin/bash", "state": "present", "uid": 1133} 2020-04-10T06:59:04.056 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313', u'name': u'nwatkins'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1133, "groups": "sudo", "home": "/home/nwatkins", "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "move_home": false, "name": "nwatkins", "shell": "/bin/bash", "state": "present", "uid": 1133} 2020-04-10T06:59:04.141 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22', u'name': u'mkidd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1134, "groups": "sudo", "home": "/home/mkidd", "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "move_home": false, "name": "mkidd", "shell": "/bin/bash", "state": "present", "uid": 1134} 2020-04-10T06:59:04.194 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22', u'name': u'mkidd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1134, "groups": "sudo", "home": "/home/mkidd", "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "move_home": false, "name": "mkidd", "shell": "/bin/bash", "state": "present", "uid": 1134} 2020-04-10T06:59:04.302 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af', u'name': u'jlopez'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1135, "groups": "sudo", "home": "/home/jlopez", "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "move_home": false, "name": "jlopez", "shell": "/bin/bash", "state": "present", "uid": 1135} 2020-04-10T06:59:04.363 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af', u'name': u'jlopez'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1135, "groups": "sudo", "home": "/home/jlopez", "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "move_home": false, "name": "jlopez", "shell": "/bin/bash", "state": "present", "uid": 1135} 2020-04-10T06:59:04.434 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb', u'name': u'haomaiwang'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1136, "groups": "sudo", "home": "/home/haomaiwang", "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "move_home": false, "name": "haomaiwang", "shell": "/bin/bash", "state": "present", "uid": 1136} 2020-04-10T06:59:04.519 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb', u'name': u'haomaiwang'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1136, "groups": "sudo", "home": "/home/haomaiwang", "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "move_home": false, "name": "haomaiwang", "shell": "/bin/bash", "state": "present", "uid": 1136} 2020-04-10T06:59:04.580 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608', u'name': u'jdillaman'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1137, "groups": "sudo", "home": "/home/jdillaman", "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "move_home": false, "name": "jdillaman", "shell": "/bin/bash", "state": "present", "uid": 1137} 2020-04-10T06:59:04.655 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608', u'name': u'jdillaman'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1137, "groups": "sudo", "home": "/home/jdillaman", "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "move_home": false, "name": "jdillaman", "shell": "/bin/bash", "state": "present", "uid": 1137} 2020-04-10T06:59:04.722 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3', u'name': u'kchai'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1138, "groups": "sudo", "home": "/home/kchai", "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "move_home": false, "name": "kchai", "shell": "/bin/bash", "state": "present", "uid": 1138} 2020-04-10T06:59:04.804 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3', u'name': u'kchai'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1138, "groups": "sudo", "home": "/home/kchai", "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "move_home": false, "name": "kchai", "shell": "/bin/bash", "state": "present", "uid": 1138} 2020-04-10T06:59:04.881 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1139} 2020-04-10T06:59:04.987 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1139} 2020-04-10T06:59:05.037 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n', u'name': u'dfuller'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/dfuller", "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "move_home": false, "name": "dfuller", "shell": "/bin/bash", "state": "present", "uid": 1140} 2020-04-10T06:59:05.133 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n', u'name': u'dfuller'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/dfuller", "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "move_home": false, "name": "dfuller", "shell": "/bin/bash", "state": "present", "uid": 1140} 2020-04-10T06:59:05.176 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n', u'name': u'owasserm'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1141, "groups": "sudo", "home": "/home/owasserm", "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "move_home": false, "name": "owasserm", "shell": "/bin/bash", "state": "present", "uid": 1141} 2020-04-10T06:59:05.283 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n', u'name': u'owasserm'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1141, "groups": "sudo", "home": "/home/owasserm", "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "move_home": false, "name": "owasserm", "shell": "/bin/bash", "state": "present", "uid": 1141} 2020-04-10T06:59:05.315 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n', u'name': u'vasu'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/vasu", "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "move_home": false, "name": "vasu", "shell": "/bin/bash", "state": "present", "uid": 1142} 2020-04-10T06:59:05.426 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n', u'name': u'vasu'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/vasu", "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "move_home": false, "name": "vasu", "shell": "/bin/bash", "state": "present", "uid": 1142} 2020-04-10T06:59:05.447 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37', u'name': u'smohan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1143, "groups": "sudo", "home": "/home/smohan", "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "move_home": false, "name": "smohan", "shell": "/bin/bash", "state": "present", "uid": 1143} 2020-04-10T06:59:05.572 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37', u'name': u'smohan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1143, "groups": "sudo", "home": "/home/smohan", "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "move_home": false, "name": "smohan", "shell": "/bin/bash", "state": "present", "uid": 1143} 2020-04-10T06:59:05.614 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114', u'name': u'abhi'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1144, "groups": "sudo", "home": "/home/abhi", "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "move_home": false, "name": "abhi", "shell": "/bin/bash", "state": "present", "uid": 1144} 2020-04-10T06:59:05.717 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114', u'name': u'abhi'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1144, "groups": "sudo", "home": "/home/abhi", "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "move_home": false, "name": "abhi", "shell": "/bin/bash", "state": "present", "uid": 1144} 2020-04-10T06:59:05.762 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611', u'name': u'cbodley'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1145, "groups": "sudo", "home": "/home/cbodley", "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "move_home": false, "name": "cbodley", "shell": "/bin/bash", "state": "present", "uid": 1145} 2020-04-10T06:59:05.867 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611', u'name': u'cbodley'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1145, "groups": "sudo", "home": "/home/cbodley", "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "move_home": false, "name": "cbodley", "shell": "/bin/bash", "state": "present", "uid": 1145} 2020-04-10T06:59:05.899 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195', u'name': u'mwatts'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1146, "groups": "sudo", "home": "/home/mwatts", "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "move_home": false, "name": "mwatts", "shell": "/bin/bash", "state": "present", "uid": 1146} 2020-04-10T06:59:06.027 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195', u'name': u'mwatts'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1146, "groups": "sudo", "home": "/home/mwatts", "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "move_home": false, "name": "mwatts", "shell": "/bin/bash", "state": "present", "uid": 1146} 2020-04-10T06:59:06.028 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1147} 2020-04-10T06:59:06.175 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1147} 2020-04-10T06:59:06.176 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea', u'name': u'pdonnell'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/pdonnell", "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "move_home": false, "name": "pdonnell", "shell": "/bin/bash", "state": "present", "uid": 1148} 2020-04-10T06:59:06.315 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea', u'name': u'pdonnell'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/pdonnell", "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "move_home": false, "name": "pdonnell", "shell": "/bin/bash", "state": "present", "uid": 1148} 2020-04-10T06:59:06.329 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868', u'name': u'jlayton'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1149, "groups": "sudo", "home": "/home/jlayton", "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "move_home": false, "name": "jlayton", "shell": "/bin/bash", "state": "present", "uid": 1149} 2020-04-10T06:59:06.470 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f', u'name': u'rzarzynski'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1150, "groups": "sudo", "home": "/home/rzarzynski", "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "move_home": false, "name": "rzarzynski", "shell": "/bin/bash", "state": "present", "uid": 1150} 2020-04-10T06:59:06.471 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868', u'name': u'jlayton'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1149, "groups": "sudo", "home": "/home/jlayton", "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "move_home": false, "name": "jlayton", "shell": "/bin/bash", "state": "present", "uid": 1149} 2020-04-10T06:59:06.610 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e', u'name': u'rdias'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1151, "groups": "sudo", "home": "/home/rdias", "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "move_home": false, "name": "rdias", "shell": "/bin/bash", "state": "present", "uid": 1151} 2020-04-10T06:59:06.622 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f', u'name': u'rzarzynski'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1150, "groups": "sudo", "home": "/home/rzarzynski", "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "move_home": false, "name": "rzarzynski", "shell": "/bin/bash", "state": "present", "uid": 1150} 2020-04-10T06:59:06.739 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953', u'name': u'vshankar'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1152, "groups": "sudo", "home": "/home/vshankar", "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "move_home": false, "name": "vshankar", "shell": "/bin/bash", "state": "present", "uid": 1152} 2020-04-10T06:59:06.761 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e', u'name': u'rdias'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1151, "groups": "sudo", "home": "/home/rdias", "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "move_home": false, "name": "rdias", "shell": "/bin/bash", "state": "present", "uid": 1151} 2020-04-10T06:59:06.892 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58', u'name': u'akupczyk'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1153, "groups": "sudo", "home": "/home/akupczyk", "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "move_home": false, "name": "akupczyk", "shell": "/bin/bash", "state": "present", "uid": 1153} 2020-04-10T06:59:06.903 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953', u'name': u'vshankar'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1152, "groups": "sudo", "home": "/home/vshankar", "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "move_home": false, "name": "vshankar", "shell": "/bin/bash", "state": "present", "uid": 1152} 2020-04-10T06:59:07.016 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n', u'name': u'nojha'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1154, "groups": "sudo", "home": "/home/nojha", "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "move_home": false, "name": "nojha", "shell": "/bin/bash", "state": "present", "uid": 1154} 2020-04-10T06:59:07.079 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58', u'name': u'akupczyk'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1153, "groups": "sudo", "home": "/home/akupczyk", "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "move_home": false, "name": "akupczyk", "shell": "/bin/bash", "state": "present", "uid": 1153} 2020-04-10T06:59:07.178 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43', u'name': u'ifed01'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/ifed01", "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "move_home": false, "name": "ifed01", "shell": "/bin/bash", "state": "present", "uid": 1155} 2020-04-10T06:59:07.236 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n', u'name': u'nojha'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1154, "groups": "sudo", "home": "/home/nojha", "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "move_home": false, "name": "nojha", "shell": "/bin/bash", "state": "present", "uid": 1154} 2020-04-10T06:59:07.327 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4', u'name': u'myoungwon'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1156, "groups": "sudo", "home": "/home/myoungwon", "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "move_home": false, "name": "myoungwon", "shell": "/bin/bash", "state": "present", "uid": 1156} 2020-04-10T06:59:07.392 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43', u'name': u'ifed01'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/ifed01", "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "move_home": false, "name": "ifed01", "shell": "/bin/bash", "state": "present", "uid": 1155} 2020-04-10T06:59:07.467 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2', u'name': u'gabrioux'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1157, "groups": "sudo", "home": "/home/gabrioux", "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "move_home": false, "name": "gabrioux", "shell": "/bin/bash", "state": "present", "uid": 1157} 2020-04-10T06:59:07.541 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4', u'name': u'myoungwon'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1156, "groups": "sudo", "home": "/home/myoungwon", "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "move_home": false, "name": "myoungwon", "shell": "/bin/bash", "state": "present", "uid": 1156} 2020-04-10T06:59:07.598 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f', u'name': u'leseb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1158, "groups": "sudo", "home": "/home/leseb", "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "move_home": false, "name": "leseb", "shell": "/bin/bash", "state": "present", "uid": 1158} 2020-04-10T06:59:07.704 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2', u'name': u'gabrioux'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1157, "groups": "sudo", "home": "/home/gabrioux", "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "move_home": false, "name": "gabrioux", "shell": "/bin/bash", "state": "present", "uid": 1157} 2020-04-10T06:59:07.754 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca', u'name': u'hchen'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1159, "groups": "sudo", "home": "/home/hchen", "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "move_home": false, "name": "hchen", "shell": "/bin/bash", "state": "present", "uid": 1159} 2020-04-10T06:59:07.848 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f', u'name': u'leseb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1158, "groups": "sudo", "home": "/home/leseb", "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "move_home": false, "name": "leseb", "shell": "/bin/bash", "state": "present", "uid": 1158} 2020-04-10T06:59:07.881 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n', u'name': u'jcollin'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1160, "groups": "sudo", "home": "/home/jcollin", "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "move_home": false, "name": "jcollin", "shell": "/bin/bash", "state": "present", "uid": 1160} 2020-04-10T06:59:07.998 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca', u'name': u'hchen'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1159, "groups": "sudo", "home": "/home/hchen", "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "move_home": false, "name": "hchen", "shell": "/bin/bash", "state": "present", "uid": 1159} 2020-04-10T06:59:08.020 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc', u'name': u'xxg'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1161, "groups": "sudo", "home": "/home/xxg", "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "move_home": false, "name": "xxg", "shell": "/bin/bash", "state": "present", "uid": 1161} 2020-04-10T06:59:08.156 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53', u'name': u'pcuzner'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1162, "groups": "sudo", "home": "/home/pcuzner", "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "move_home": false, "name": "pcuzner", "shell": "/bin/bash", "state": "present", "uid": 1162} 2020-04-10T06:59:08.163 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n', u'name': u'jcollin'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1160, "groups": "sudo", "home": "/home/jcollin", "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "move_home": false, "name": "jcollin", "shell": "/bin/bash", "state": "present", "uid": 1160} 2020-04-10T06:59:08.291 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2', u'name': u'mkogan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1163, "groups": "sudo", "home": "/home/mkogan", "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "move_home": false, "name": "mkogan", "shell": "/bin/bash", "state": "present", "uid": 1163} 2020-04-10T06:59:08.304 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc', u'name': u'xxg'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1161, "groups": "sudo", "home": "/home/xxg", "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "move_home": false, "name": "xxg", "shell": "/bin/bash", "state": "present", "uid": 1161} 2020-04-10T06:59:08.432 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1139} 2020-04-10T06:59:08.434 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53', u'name': u'pcuzner'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1162, "groups": "sudo", "home": "/home/pcuzner", "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "move_home": false, "name": "pcuzner", "shell": "/bin/bash", "state": "present", "uid": 1162} 2020-04-10T06:59:08.574 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736', u'name': u'henrix'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1164, "groups": "sudo", "home": "/home/henrix", "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "move_home": false, "name": "henrix", "shell": "/bin/bash", "state": "present", "uid": 1164} 2020-04-10T06:59:08.607 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2', u'name': u'mkogan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1163, "groups": "sudo", "home": "/home/mkogan", "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "move_home": false, "name": "mkogan", "shell": "/bin/bash", "state": "present", "uid": 1163} 2020-04-10T06:59:08.705 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26', u'name': u'pbs1108'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1165, "groups": "sudo", "home": "/home/pbs1108", "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "move_home": false, "name": "pbs1108", "shell": "/bin/bash", "state": "present", "uid": 1165} 2020-04-10T06:59:08.758 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1139} 2020-04-10T06:59:08.854 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315', u'name': u'clacroix'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1166, "groups": "sudo", "home": "/home/clacroix", "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "move_home": false, "name": "clacroix", "shell": "/bin/bash", "state": "present", "uid": 1166} 2020-04-10T06:59:08.908 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736', u'name': u'henrix'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1164, "groups": "sudo", "home": "/home/henrix", "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "move_home": false, "name": "henrix", "shell": "/bin/bash", "state": "present", "uid": 1164} 2020-04-10T06:59:08.993 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006', u'name': u'epuertat'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1167, "groups": "sudo", "home": "/home/epuertat", "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "move_home": false, "name": "epuertat", "shell": "/bin/bash", "state": "present", "uid": 1167} 2020-04-10T06:59:09.047 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26', u'name': u'pbs1108'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1165, "groups": "sudo", "home": "/home/pbs1108", "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "move_home": false, "name": "pbs1108", "shell": "/bin/bash", "state": "present", "uid": 1165} 2020-04-10T06:59:09.193 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315', u'name': u'clacroix'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1166, "groups": "sudo", "home": "/home/clacroix", "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "move_home": false, "name": "clacroix", "shell": "/bin/bash", "state": "present", "uid": 1166} 2020-04-10T06:59:09.264 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461', u'name': u'tdehler'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1168, "groups": "sudo", "home": "/home/tdehler", "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "move_home": false, "name": "tdehler", "shell": "/bin/bash", "state": "present", "uid": 1168} 2020-04-10T06:59:09.329 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006', u'name': u'epuertat'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1167, "groups": "sudo", "home": "/home/epuertat", "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "move_home": false, "name": "epuertat", "shell": "/bin/bash", "state": "present", "uid": 1167} 2020-04-10T06:59:09.398 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305', u'name': u'laura'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1169, "groups": "sudo", "home": "/home/laura", "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "move_home": false, "name": "laura", "shell": "/bin/bash", "state": "present", "uid": 1169} 2020-04-10T06:59:09.545 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce', u'name': u'yaarit'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1170, "groups": "sudo", "home": "/home/yaarit", "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "move_home": false, "name": "yaarit", "shell": "/bin/bash", "state": "present", "uid": 1170} 2020-04-10T06:59:09.622 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461', u'name': u'tdehler'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1168, "groups": "sudo", "home": "/home/tdehler", "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "move_home": false, "name": "tdehler", "shell": "/bin/bash", "state": "present", "uid": 1168} 2020-04-10T06:59:09.676 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3', u'name': u'rpavani1998'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1171, "groups": "sudo", "home": "/home/rpavani1998", "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "move_home": false, "name": "rpavani1998", "shell": "/bin/bash", "state": "present", "uid": 1171} 2020-04-10T06:59:09.771 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305', u'name': u'laura'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1169, "groups": "sudo", "home": "/home/laura", "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "move_home": false, "name": "laura", "shell": "/bin/bash", "state": "present", "uid": 1169} 2020-04-10T06:59:09.814 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4', u'name': u'rishabh'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1172, "groups": "sudo", "home": "/home/rishabh", "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "move_home": false, "name": "rishabh", "shell": "/bin/bash", "state": "present", "uid": 1172} 2020-04-10T06:59:09.908 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce', u'name': u'yaarit'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1170, "groups": "sudo", "home": "/home/yaarit", "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "move_home": false, "name": "yaarit", "shell": "/bin/bash", "state": "present", "uid": 1170} 2020-04-10T06:59:09.941 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2', u'name': u'skrah'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1173, "groups": "sudo", "home": "/home/skrah", "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "move_home": false, "name": "skrah", "shell": "/bin/bash", "state": "present", "uid": 1173} 2020-04-10T06:59:10.105 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8', u'name': u'smanjara'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1174, "groups": "sudo", "home": "/home/smanjara", "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "move_home": false, "name": "smanjara", "shell": "/bin/bash", "state": "present", "uid": 1174} 2020-04-10T06:59:10.109 INFO:teuthology.task.ansible.out: 2020-04-10T06:59:10.132 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3', u'name': u'rpavani1998'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1171, "groups": "sudo", "home": "/home/rpavani1998", "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "move_home": false, "name": "rpavani1998", "shell": "/bin/bash", "state": "present", "uid": 1171} 2020-04-10T06:59:10.280 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41', u'name': u'bengland'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1175, "groups": "sudo", "home": "/home/bengland", "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "move_home": false, "name": "bengland", "shell": "/bin/bash", "state": "present", "uid": 1175} 2020-04-10T06:59:10.281 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4', u'name': u'rishabh'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1172, "groups": "sudo", "home": "/home/rishabh", "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "move_home": false, "name": "rishabh", "shell": "/bin/bash", "state": "present", "uid": 1172} 2020-04-10T06:59:10.412 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d', u'name': u'pnawracay'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1176, "groups": "sudo", "home": "/home/pnawracay", "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "move_home": false, "name": "pnawracay", "shell": "/bin/bash", "state": "present", "uid": 1176} 2020-04-10T06:59:10.445 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2', u'name': u'skrah'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1173, "groups": "sudo", "home": "/home/skrah", "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "move_home": false, "name": "skrah", "shell": "/bin/bash", "state": "present", "uid": 1173} 2020-04-10T06:59:10.541 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8', u'name': u'alfonsomthd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1177, "groups": "sudo", "home": "/home/alfonsomthd", "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "move_home": false, "name": "alfonsomthd", "shell": "/bin/bash", "state": "present", "uid": 1177} 2020-04-10T06:59:10.585 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8', u'name': u'smanjara'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1174, "groups": "sudo", "home": "/home/smanjara", "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "move_home": false, "name": "smanjara", "shell": "/bin/bash", "state": "present", "uid": 1174} 2020-04-10T06:59:10.675 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e', u'name': u'oliveiradan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1178, "groups": "sudo", "home": "/home/oliveiradan", "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "move_home": false, "name": "oliveiradan", "shell": "/bin/bash", "state": "present", "uid": 1178} 2020-04-10T06:59:10.708 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41', u'name': u'bengland'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1175, "groups": "sudo", "home": "/home/bengland", "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "move_home": false, "name": "bengland", "shell": "/bin/bash", "state": "present", "uid": 1175} 2020-04-10T06:59:10.815 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba', u'name': u'swagner'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1179, "groups": "sudo", "home": "/home/swagner", "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "move_home": false, "name": "swagner", "shell": "/bin/bash", "state": "present", "uid": 1179} 2020-04-10T06:59:10.869 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d', u'name': u'pnawracay'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1176, "groups": "sudo", "home": "/home/pnawracay", "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "move_home": false, "name": "pnawracay", "shell": "/bin/bash", "state": "present", "uid": 1176} 2020-04-10T06:59:10.955 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e', u'name': u'yuvalif'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1180, "groups": "sudo", "home": "/home/yuvalif", "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "move_home": false, "name": "yuvalif", "shell": "/bin/bash", "state": "present", "uid": 1180} 2020-04-10T06:59:11.019 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8', u'name': u'alfonsomthd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1177, "groups": "sudo", "home": "/home/alfonsomthd", "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "move_home": false, "name": "alfonsomthd", "shell": "/bin/bash", "state": "present", "uid": 1177} 2020-04-10T06:59:11.082 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852', u'name': u'kkeithle'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1181, "groups": "sudo", "home": "/home/kkeithle", "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "move_home": false, "name": "kkeithle", "shell": "/bin/bash", "state": "present", "uid": 1181} 2020-04-10T06:59:11.168 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e', u'name': u'oliveiradan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1178, "groups": "sudo", "home": "/home/oliveiradan", "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "move_home": false, "name": "oliveiradan", "shell": "/bin/bash", "state": "present", "uid": 1178} 2020-04-10T06:59:11.211 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130', u'name': u'emmericp'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1182, "groups": "sudo", "home": "/home/emmericp", "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "move_home": false, "name": "emmericp", "shell": "/bin/bash", "state": "present", "uid": 1182} 2020-04-10T06:59:11.306 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba', u'name': u'swagner'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1179, "groups": "sudo", "home": "/home/swagner", "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "move_home": false, "name": "swagner", "shell": "/bin/bash", "state": "present", "uid": 1179} 2020-04-10T06:59:11.350 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0', u'name': u'mchangir'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1183, "groups": "sudo", "home": "/home/mchangir", "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "move_home": false, "name": "mchangir", "shell": "/bin/bash", "state": "present", "uid": 1183} 2020-04-10T06:59:11.442 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e', u'name': u'yuvalif'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1180, "groups": "sudo", "home": "/home/yuvalif", "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "move_home": false, "name": "yuvalif", "shell": "/bin/bash", "state": "present", "uid": 1180} 2020-04-10T06:59:11.485 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286', u'name': u'sidharthanup'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1184, "groups": "sudo", "home": "/home/sidharthanup", "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "move_home": false, "name": "sidharthanup", "shell": "/bin/bash", "state": "present", "uid": 1184} 2020-04-10T06:59:11.592 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852', u'name': u'kkeithle'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1181, "groups": "sudo", "home": "/home/kkeithle", "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "move_home": false, "name": "kkeithle", "shell": "/bin/bash", "state": "present", "uid": 1181} 2020-04-10T06:59:11.615 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31', u'name': u'varsha'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1185, "groups": "sudo", "home": "/home/varsha", "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "move_home": false, "name": "varsha", "shell": "/bin/bash", "state": "present", "uid": 1185} 2020-04-10T06:59:11.754 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130', u'name': u'emmericp'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1182, "groups": "sudo", "home": "/home/emmericp", "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "move_home": false, "name": "emmericp", "shell": "/bin/bash", "state": "present", "uid": 1182} 2020-04-10T06:59:11.766 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n', u'name': u'sjust'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1186, "groups": "sudo", "home": "/home/sjust", "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "move_home": false, "name": "sjust", "shell": "/bin/bash", "state": "present", "uid": 1186} 2020-04-10T06:59:11.890 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0', u'name': u'mchangir'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1183, "groups": "sudo", "home": "/home/mchangir", "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "move_home": false, "name": "mchangir", "shell": "/bin/bash", "state": "present", "uid": 1183} 2020-04-10T06:59:11.900 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'deepika@x1carbon BgvhtHMrCcelT98ekPDMoA 6fa3cf08186b93b966b68841f3507a0809d348c5097b39b6140832cac2afd511', u'name': u'ideepika'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1187, "groups": "sudo", "home": "/home/ideepika", "item": {"name": "ideepika", "ovpn": "deepika@x1carbon BgvhtHMrCcelT98ekPDMoA 6fa3cf08186b93b966b68841f3507a0809d348c5097b39b6140832cac2afd511"}, "move_home": false, "name": "ideepika", "shell": "/bin/bash", "state": "present", "uid": 1187} 2020-04-10T06:59:12.032 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286', u'name': u'sidharthanup'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1184, "groups": "sudo", "home": "/home/sidharthanup", "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "move_home": false, "name": "sidharthanup", "shell": "/bin/bash", "state": "present", "uid": 1184} ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46', u'name': u'gsalomon'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1188, "groups": "sudo", "home": "/home/gsalomon", "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "move_home": false, "name": "gsalomon", "shell": "/bin/bash", "state": "present", "uid": 1188} 2020-04-10T06:59:12.168 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de', u'name': u'soumyakoduri'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1189, "groups": "sudo", "home": "/home/soumyakoduri", "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "move_home": false, "name": "soumyakoduri", "shell": "/bin/bash", "state": "present", "uid": 1189} 2020-04-10T06:59:12.180 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31', u'name': u'varsha'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1185, "groups": "sudo", "home": "/home/varsha", "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "move_home": false, "name": "varsha", "shell": "/bin/bash", "state": "present", "uid": 1185} 2020-04-10T06:59:12.308 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a', u'name': u'kyr'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1190, "groups": "sudo", "home": "/home/kyr", "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "move_home": false, "name": "kyr", "shell": "/bin/bash", "state": "present", "uid": 1190} 2020-04-10T06:59:12.330 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n', u'name': u'sjust'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1186, "groups": "sudo", "home": "/home/sjust", "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "move_home": false, "name": "sjust", "shell": "/bin/bash", "state": "present", "uid": 1186} 2020-04-10T06:59:12.448 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849', u'name': u'sseshasa'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1191, "groups": "sudo", "home": "/home/sseshasa", "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "move_home": false, "name": "sseshasa", "shell": "/bin/bash", "state": "present", "uid": 1191} 2020-04-10T06:59:12.505 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'deepika@x1carbon BgvhtHMrCcelT98ekPDMoA 6fa3cf08186b93b966b68841f3507a0809d348c5097b39b6140832cac2afd511', u'name': u'ideepika'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1187, "groups": "sudo", "home": "/home/ideepika", "item": {"name": "ideepika", "ovpn": "deepika@x1carbon BgvhtHMrCcelT98ekPDMoA 6fa3cf08186b93b966b68841f3507a0809d348c5097b39b6140832cac2afd511"}, "move_home": false, "name": "ideepika", "shell": "/bin/bash", "state": "present", "uid": 1187} 2020-04-10T06:59:12.573 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189', u'name': u'rfriedma'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1192, "groups": "sudo", "home": "/home/rfriedma", "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "move_home": false, "name": "rfriedma", "shell": "/bin/bash", "state": "present", "uid": 1192} 2020-04-10T06:59:12.615 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46', u'name': u'gsalomon'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1188, "groups": "sudo", "home": "/home/gsalomon", "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "move_home": false, "name": "gsalomon", "shell": "/bin/bash", "state": "present", "uid": 1188} 2020-04-10T06:59:12.699 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b', u'name': u'lmb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1193, "groups": "sudo", "home": "/home/lmb", "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "move_home": false, "name": "lmb", "shell": "/bin/bash", "state": "present", "uid": 1193} 2020-04-10T06:59:12.794 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de', u'name': u'soumyakoduri'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1189, "groups": "sudo", "home": "/home/soumyakoduri", "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "move_home": false, "name": "soumyakoduri", "shell": "/bin/bash", "state": "present", "uid": 1189} 2020-04-10T06:59:12.827 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc', u'name': u'ksirivad'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1194, "groups": "sudo", "home": "/home/ksirivad", "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "move_home": false, "name": "ksirivad", "shell": "/bin/bash", "state": "present", "uid": 1194} 2020-04-10T06:59:12.923 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a', u'name': u'kyr'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1190, "groups": "sudo", "home": "/home/kyr", "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "move_home": false, "name": "kyr", "shell": "/bin/bash", "state": "present", "uid": 1190} 2020-04-10T06:59:12.969 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4', u'name': u'ofriedma'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1195, "groups": "sudo", "home": "/home/ofriedma", "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "move_home": false, "name": "ofriedma", "shell": "/bin/bash", "state": "present", "uid": 1195} 2020-04-10T06:59:13.074 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849', u'name': u'sseshasa'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1191, "groups": "sudo", "home": "/home/sseshasa", "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "move_home": false, "name": "sseshasa", "shell": "/bin/bash", "state": "present", "uid": 1191} 2020-04-10T06:59:13.107 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52', u'name': u'h4iresis'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1196, "groups": "sudo", "home": "/home/h4iresis", "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "move_home": false, "name": "h4iresis", "shell": "/bin/bash", "state": "present", "uid": 1196} 2020-04-10T06:59:13.224 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189', u'name': u'rfriedma'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1192, "groups": "sudo", "home": "/home/rfriedma", "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "move_home": false, "name": "rfriedma", "shell": "/bin/bash", "state": "present", "uid": 1192} 2020-04-10T06:59:13.265 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5', u'name': u'shyukri'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1197, "groups": "sudo", "home": "/home/shyukri", "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "move_home": false, "name": "shyukri", "shell": "/bin/bash", "state": "present", "uid": 1197} 2020-04-10T06:59:13.366 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b', u'name': u'lmb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1193, "groups": "sudo", "home": "/home/lmb", "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "move_home": false, "name": "lmb", "shell": "/bin/bash", "state": "present", "uid": 1193} 2020-04-10T06:59:13.388 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e', u'name': u'scarvalhojr'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1198, "groups": "sudo", "home": "/home/scarvalhojr", "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "move_home": false, "name": "scarvalhojr", "shell": "/bin/bash", "state": "present", "uid": 1198} 2020-04-10T06:59:13.515 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc', u'name': u'ksirivad'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1194, "groups": "sudo", "home": "/home/ksirivad", "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "move_home": false, "name": "ksirivad", "shell": "/bin/bash", "state": "present", "uid": 1194} 2020-04-10T06:59:13.527 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b', u'name': u'rakesh'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1199, "groups": "sudo", "home": "/home/rakesh", "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "move_home": false, "name": "rakesh", "shell": "/bin/bash", "state": "present", "uid": 1199} 2020-04-10T06:59:13.663 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4', u'name': u'ofriedma'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1195, "groups": "sudo", "home": "/home/ofriedma", "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "move_home": false, "name": "ofriedma", "shell": "/bin/bash", "state": "present", "uid": 1195} 2020-04-10T06:59:13.684 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f', u'name': u'ryg-'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1200, "groups": "sudo", "home": "/home/ryg-", "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "move_home": false, "name": "ryg-", "shell": "/bin/bash", "state": "present", "uid": 1200} 2020-04-10T06:59:13.822 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1147} 2020-04-10T06:59:13.834 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52', u'name': u'h4iresis'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1196, "groups": "sudo", "home": "/home/h4iresis", "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "move_home": false, "name": "h4iresis", "shell": "/bin/bash", "state": "present", "uid": 1196} 2020-04-10T06:59:13.949 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42', u'name': u'zdover'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1201, "groups": "sudo", "home": "/home/zdover", "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "move_home": false, "name": "zdover", "shell": "/bin/bash", "state": "present", "uid": 1201} 2020-04-10T06:59:13.972 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5', u'name': u'shyukri'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1197, "groups": "sudo", "home": "/home/shyukri", "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "move_home": false, "name": "shyukri", "shell": "/bin/bash", "state": "present", "uid": 1197} 2020-04-10T06:59:14.086 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b', u'name': u'rraja'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1202, "groups": "sudo", "home": "/home/rraja", "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "move_home": false, "name": "rraja", "shell": "/bin/bash", "state": "present", "uid": 1202} 2020-04-10T06:59:14.118 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e', u'name': u'scarvalhojr'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1198, "groups": "sudo", "home": "/home/scarvalhojr", "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "move_home": false, "name": "scarvalhojr", "shell": "/bin/bash", "state": "present", "uid": 1198} 2020-04-10T06:59:14.238 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f', u'name': u'toabctl'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1203, "groups": "sudo", "home": "/home/toabctl", "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "move_home": false, "name": "toabctl", "shell": "/bin/bash", "state": "present", "uid": 1203} 2020-04-10T06:59:14.281 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b', u'name': u'rakesh'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1199, "groups": "sudo", "home": "/home/rakesh", "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "move_home": false, "name": "rakesh", "shell": "/bin/bash", "state": "present", "uid": 1199} 2020-04-10T06:59:14.365 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39', u'name': u'mgfritch'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1204, "groups": "sudo", "home": "/home/mgfritch", "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "move_home": false, "name": "mgfritch", "shell": "/bin/bash", "state": "present", "uid": 1204} 2020-04-10T06:59:14.428 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f', u'name': u'ryg-'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1200, "groups": "sudo", "home": "/home/ryg-", "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "move_home": false, "name": "ryg-", "shell": "/bin/bash", "state": "present", "uid": 1200} 2020-04-10T06:59:14.504 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'xiubli@fedora1 vosB15bA3uALkBrmjcKFUg 4ef5cb132c30563fd5fb9e4e9352786ad0d2258e41ed63f91c1f65f69b54d50d', u'name': u'xiubli'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1205, "groups": "sudo", "home": "/home/xiubli", "item": {"name": "xiubli", "ovpn": "xiubli@fedora1 vosB15bA3uALkBrmjcKFUg 4ef5cb132c30563fd5fb9e4e9352786ad0d2258e41ed63f91c1f65f69b54d50d"}, "move_home": false, "name": "xiubli", "shell": "/bin/bash", "state": "present", "uid": 1205} 2020-04-10T06:59:14.581 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1147} 2020-04-10T06:59:14.645 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f', u'name': u'jafaj'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1206, "groups": "sudo", "home": "/home/jafaj", "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "move_home": false, "name": "jafaj", "shell": "/bin/bash", "state": "present", "uid": 1206} 2020-04-10T06:59:14.740 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42', u'name': u'zdover'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1201, "groups": "sudo", "home": "/home/zdover", "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "move_home": false, "name": "zdover", "shell": "/bin/bash", "state": "present", "uid": 1201} 2020-04-10T06:59:14.783 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef', u'name': u'jschmid'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1207, "groups": "sudo", "home": "/home/jschmid", "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "move_home": false, "name": "jschmid", "shell": "/bin/bash", "state": "present", "uid": 1207} 2020-04-10T06:59:14.883 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b', u'name': u'rraja'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1202, "groups": "sudo", "home": "/home/rraja", "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "move_home": false, "name": "rraja", "shell": "/bin/bash", "state": "present", "uid": 1202} 2020-04-10T06:59:14.915 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'kachau@thinkpad GsZIGCdLojpvOZDxwcWSlg 0638220db65953e0cd1f134c499e4a713b32a882c8a92458afcb495dbd087518', u'name': u'kachau'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1208, "groups": "sudo", "home": "/home/kachau", "item": {"name": "kachau", "ovpn": "kachau@thinkpad GsZIGCdLojpvOZDxwcWSlg 0638220db65953e0cd1f134c499e4a713b32a882c8a92458afcb495dbd087518"}, "move_home": false, "name": "kachau", "shell": "/bin/bash", "state": "present", "uid": 1208} 2020-04-10T06:59:15.018 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f', u'name': u'toabctl'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1203, "groups": "sudo", "home": "/home/toabctl", "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "move_home": false, "name": "toabctl", "shell": "/bin/bash", "state": "present", "uid": 1203} 2020-04-10T06:59:15.050 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034', u'name': u'khiremat'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1209, "groups": "sudo", "home": "/home/khiremat", "item": {"name": "khiremat", "ovpn": "khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034"}, "move_home": false, "name": "khiremat", "shell": "/bin/bash", "state": "present", "uid": 1209} 2020-04-10T06:59:15.166 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39', u'name': u'mgfritch'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1204, "groups": "sudo", "home": "/home/mgfritch", "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "move_home": false, "name": "mgfritch", "shell": "/bin/bash", "state": "present", "uid": 1204} 2020-04-10T06:59:15.188 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'kapandya@thinkpad Uw1JHj0kPIqSd9iOJ68TNg 46badc3b5cd150bfb74daf4300c8329491000521738ba70455631aa4a47e42c0', u'name': u'kapandya'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1210, "groups": "sudo", "home": "/home/kapandya", "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad Uw1JHj0kPIqSd9iOJ68TNg 46badc3b5cd150bfb74daf4300c8329491000521738ba70455631aa4a47e42c0"}, "move_home": false, "name": "kapandya", "shell": "/bin/bash", "state": "present", "uid": 1210} 2020-04-10T06:59:15.307 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'xiubli@fedora1 vosB15bA3uALkBrmjcKFUg 4ef5cb132c30563fd5fb9e4e9352786ad0d2258e41ed63f91c1f65f69b54d50d', u'name': u'xiubli'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1205, "groups": "sudo", "home": "/home/xiubli", "item": {"name": "xiubli", "ovpn": "xiubli@fedora1 vosB15bA3uALkBrmjcKFUg 4ef5cb132c30563fd5fb9e4e9352786ad0d2258e41ed63f91c1f65f69b54d50d"}, "move_home": false, "name": "xiubli", "shell": "/bin/bash", "state": "present", "uid": 1205} 2020-04-10T06:59:15.453 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f', u'name': u'jafaj'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1206, "groups": "sudo", "home": "/home/jafaj", "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "move_home": false, "name": "jafaj", "shell": "/bin/bash", "state": "present", "uid": 1206} 2020-04-10T06:59:15.589 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef', u'name': u'jschmid'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1207, "groups": "sudo", "home": "/home/jschmid", "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "move_home": false, "name": "jschmid", "shell": "/bin/bash", "state": "present", "uid": 1207} 2020-04-10T06:59:15.747 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'kachau@thinkpad GsZIGCdLojpvOZDxwcWSlg 0638220db65953e0cd1f134c499e4a713b32a882c8a92458afcb495dbd087518', u'name': u'kachau'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1208, "groups": "sudo", "home": "/home/kachau", "item": {"name": "kachau", "ovpn": "kachau@thinkpad GsZIGCdLojpvOZDxwcWSlg 0638220db65953e0cd1f134c499e4a713b32a882c8a92458afcb495dbd087518"}, "move_home": false, "name": "kachau", "shell": "/bin/bash", "state": "present", "uid": 1208} 2020-04-10T06:59:15.873 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034', u'name': u'khiremat'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1209, "groups": "sudo", "home": "/home/khiremat", "item": {"name": "khiremat", "ovpn": "khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034"}, "move_home": false, "name": "khiremat", "shell": "/bin/bash", "state": "present", "uid": 1209} 2020-04-10T06:59:15.895 INFO:teuthology.task.ansible.out:changed: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89', u'name': u'prasad'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10105, "groups": "sudo", "home": "/home/prasad", "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "name": "prasad", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10104} 2020-04-10T06:59:16.032 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:16 +0000 (0:00:17.843) 0:00:26.485 ********** 2020-04-10T06:59:16.038 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'kapandya@thinkpad Uw1JHj0kPIqSd9iOJ68TNg 46badc3b5cd150bfb74daf4300c8329491000521738ba70455631aa4a47e42c0', u'name': u'kapandya'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1210, "groups": "sudo", "home": "/home/kapandya", "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad Uw1JHj0kPIqSd9iOJ68TNg 46badc3b5cd150bfb74daf4300c8329491000521738ba70455631aa4a47e42c0"}, "move_home": false, "name": "kapandya", "shell": "/bin/bash", "state": "present", "uid": 1210} 2020-04-10T06:59:16.063 INFO:teuthology.task.ansible.out: TASK [users : Create all users without sudo access.] *************************** 2020-04-10T06:59:16.102 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:16 +0000 (0:00:00.070) 0:00:26.555 ********** 2020-04-10T06:59:16.150 INFO:teuthology.task.ansible.out: TASK [users : Merge managed_users and managed_admin users] ********************* 2020-04-10T06:59:16.155 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => {"ansible_facts": {"pubkey_users": [{"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, {"name": "jcaratzas", "ovpn": "jcaratzas@laptop 2020-04-10T06:59:16.156 INFO:teuthology.task.ansible.out: 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9"}, {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAx 2020-04-10T06:59:16.158 INFO:teuthology.task.ansible.out:j+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8ea 2020-04-10T06:59:16.159 INFO:teuthology.task.ansible.out:c8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f261 2020-04-10T06:59:16.161 INFO:teuthology.task.ansible.out:02d239d4"}, {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, {"name": "haomai 2020-04-10T06:59:16.162 INFO:teuthology.task.ansible.out:wang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, {"name": "pdonnell", "ovpn": "pdonnell@redha 2020-04-10T06:59:16.163 INFO:teuthology.task.ansible.out:t.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, {" 2020-04-10T06:59:16.164 INFO:teuthology.task.ansible.out:name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMiv 2020-04-10T06:59:16.165 INFO:teuthology.task.ansible.out:DNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, {"name": "bengland", "ovpn": "beng 2020-04-10T06:59:16.166 INFO:teuthology.task.ansible.out:land@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, {"name" 2020-04-10T06:59:16.168 INFO:teuthology.task.ansible.out:: "ideepika", "ovpn": "deepika@x1carbon BgvhtHMrCcelT98ekPDMoA 6fa3cf08186b93b966b68841f3507a0809d348c5097b39b6140832cac2afd511"}, {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643 2020-04-10T06:59:16.169 INFO:teuthology.task.ansible.out:eaca10bbf756f2b8313645d2b8da3"}, {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, {"name": "xiubli", "ovpn": "xiubli@fedora1 vosB15bA3uALkBrmjcKFUg 4ef5cb132c30563fd5fb9e4e9352786ad0d2258e41ed63f91c1f65f69b54d50d"}, {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, {"name": "kachau", "ovpn": "kachau@thinkpad GsZIGCdLojpvOZDxwcWSlg 0638220db65953e0cd1f134c499e4a713b32a882c8a92458afcb495dbd087518"}, {"name": "khiremat", "ovpn": "khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034"}, {"name": "kapandya", "ovpn": "kapandya@thinkpad Uw1JHj0kPIqSd9iOJ68TNg 46badc3b5cd150bfb74daf4300c8329491000521738ba70455631aa4a47e42c0"}, {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}]}, "changed": false} 2020-04-10T06:59:16.209 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:16 +0000 (0:00:00.106) 0:00:26.661 ********** 2020-04-10T06:59:16.564 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2020-04-10T06:59:16.564 INFO:teuthology.task.ansible.out:changed: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89', u'name': u'prasad'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10105, "groups": "sudo", "home": "/home/prasad", "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "name": "prasad", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10104} 2020-04-10T06:59:16.718 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:16 +0000 (0:00:00.509) 0:00:27.171 ********** 2020-04-10T06:59:16.745 INFO:teuthology.task.ansible.out: TASK [users : Create all users without sudo access.] *************************** 2020-04-10T06:59:16.788 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:16 +0000 (0:00:00.070) 0:00:27.242 ********** 2020-04-10T06:59:16.837 INFO:teuthology.task.ansible.out: TASK [users : Merge managed_users and managed_admin users] ********************* 2020-04-10T06:59:16.844 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => {"ansible_facts": {"pubkey_users": [{"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, {"name": "jcaratzas", "ovpn": "jcaratzas@laptop 2020-04-10T06:59:16.845 INFO:teuthology.task.ansible.out: 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9"}, {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAx 2020-04-10T06:59:16.847 INFO:teuthology.task.ansible.out:j+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8ea 2020-04-10T06:59:16.848 INFO:teuthology.task.ansible.out:c8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0 2020-04-10T06:59:16.848 INFO:teuthology.task.ansible.out:bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f261 2020-04-10T06:59:16.850 INFO:teuthology.task.ansible.out:02d239d4"}, {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, {"name": "haomai 2020-04-10T06:59:16.851 INFO:teuthology.task.ansible.out:wang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, {"name": "pdonnell", "ovpn": "pdonnell@redha 2020-04-10T06:59:16.853 INFO:teuthology.task.ansible.out:t.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMiv 2020-04-10T06:59:16.854 INFO:teuthology.task.ansible.out:DNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7b 2020-04-10T06:59:16.861 INFO:teuthology.task.ansible.out:d86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, {"name": "bengland", "ovpn": "beng 2020-04-10T06:59:16.862 INFO:teuthology.task.ansible.out:land@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, {"name" 2020-04-10T06:59:16.864 INFO:teuthology.task.ansible.out:: "ideepika", "ovpn": "deepika@x1carbon BgvhtHMrCcelT98ekPDMoA 6fa3cf08186b93b966b68841f3507a0809d348c5097b39b6140832cac2afd511"}, {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643 2020-04-10T06:59:16.865 INFO:teuthology.task.ansible.out:eaca10bbf756f2b8313645d2b8da3"}, {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, {"name": "xiubli", "ovpn": "xiubli@fedora1 vosB15bA3uALkBrmjcKFUg 4ef5cb132c30563fd5fb9e4e9352786ad0d2258e41ed63f91c1f65f69b54d50d"}, {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, {"name": "kachau", "ovpn": "kachau@thinkpad GsZIGCdLojpvOZDxwcWSlg 0638220db65953e0cd1f134c499e4a713b32a882c8a92458afcb495dbd087518"}, {"name": "khiremat", "ovpn": "khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034"}, {"name": "kapandya", "ovpn": "kapandya@thinkpad Uw1JHj0kPIqSd9iOJ68TNg 46badc3b5cd150bfb74daf4300c8329491000521738ba70455631aa4a47e42c0"}, {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}]}, "changed": false} 2020-04-10T06:59:16.912 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:16 +0000 (0:00:00.123) 0:00:27.365 ********** 2020-04-10T06:59:17.221 INFO:teuthology.task.ansible.out: TASK [users : Clone the keys repo] ********************************************* ok: [smithi101.front.sepia.ceph.com -> localhost] => {"after": "f5045e16501fc514de59d088aafd14d076981fdf", "attempts": 1, "before": "f5045e16501fc514de59d088aafd14d076981fdf", "changed": false, "remote_url_changed": false} 2020-04-10T06:59:17.266 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:17 +0000 (0:00:00.353) 0:00:27.719 ********** 2020-04-10T06:59:17.466 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9', u'name': u'andrewschoen'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com", "key_options": null, "keyfile": "/home/andrewschoen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "andrewschoen", "validate_certs": true} 2020-04-10T06:59:17.573 INFO:teuthology.task.ansible.out: TASK [users : Clone the keys repo] ********************************************* 2020-04-10T06:59:17.575 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com -> localhost] => {"after": "f5045e16501fc514de59d088aafd14d076981fdf", "attempts": 1, "before": "f5045e16501fc514de59d088aafd14d076981fdf", "changed": false, "remote_url_changed": false} 2020-04-10T06:59:17.635 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:17 +0000 (0:00:00.368) 0:00:28.088 ********** 2020-04-10T06:59:17.641 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2020-04-10T06:59:17.642 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n', u'name': u'zack'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/zack/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zack", "validate_certs": true} 2020-04-10T06:59:17.777 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd', u'name': u'kdreyer'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4eHmz10szeHNS3dNejKokW85ksB+iR4HGOFsmQM11Ni68Nm5aqEKvkOZU8TpY92vpCQL0A68GlrXB845cACdyk6HUJYyNNNMC43l1FYWOwjMqQBSdj8W3VQDTA6eiG60mt5fgI8fyR38rKzIA1MnTBkSSjuh5kQVJ9bdEp3GuY5oc8vxDNBlGJ6LYnyEWt/pqL2J+mpjqnOjsC+EbE2exhP9O+mvzpQiyo/+dEN1COwX3//pNRXGfOSeOczHNsJE8Eu+j/n/BlW57++sJyFMkzS7bUxMSGM6quvjQZ7RT1c5JM6vLEiQyzQxoRgzY93h1yKlOstBi0NamtpqHQZGP kdreyer@mbp.ktdreyer.com", "key_options": null, "keyfile": "/home/kdreyer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kdreyer", "validate_certs": true} 2020-04-10T06:59:17.837 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9', u'name': u'andrewschoen'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com", "key_options": null, "keyfile": "/home/andrewschoen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "andrewschoen", "validat 2020-04-10T06:59:17.837 INFO:teuthology.task.ansible.out:e_certs": true} 2020-04-10T06:59:17.902 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n', u'name': u'dmick'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/onMoOCUhkp+UiJAq+ye4P70sfeV+uMhgWKLfLBh/1SYq1xxb9bSmjBnETO/9KvukRhzlReBQ7y+BKRhm037P7LuTUY4r0Y8JB6BXkjjCY1yXZJC6kvB7xFsvyiDum5rrdeWMQ/Rhh7YJpze5KY1nHLkyU9X7Ewak0kLEyduJEKekbsOom6l0wZ2ZYrBEKFWJXUDk5icUuWWg3mhKaKnLNWVg1lUw8jzYTJYfjR9GUgzZWXtzkAaI3ugpLOlQ+KEls03/WAFcWA9F9ExqzIsJpvEks27LeFBGVE3PjVNKMqqB 2020-04-10T06:59:17.904 INFO:teuthology.task.ansible.out:ZQlZtIEz1iguWJaWMi5lDIfXkvVOFabtIpX/BZRf dmick@dmick-MacBookAir\nssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOoIm6sOytKwqNf9FSQn0Olu72ZRDjAYeNFQVWTn2PIB ed25519-hplaptop", "key_options": null, "keyfile": "/home/dmick/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dmick", "validate_certs": true} 2020-04-10T06:59:17.966 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n', u'name': u'zack'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/zack/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zack", "validate_certs": true} 2020-04-10T06:59:18.039 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n', u'name': u'sage'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJ4z0gLwjexgyixdTuiwaxpPnq7w4FOKxHsQKKAHTlKRu11ynB+9APbtqaAYss8yDqHaHyewPdE+9JMYhZX8rUgcKv2H7jBCQ/Enfl0UyY643VQnuG8lr2Ipk6ELuL5NuuTb7Npo4zKyq/0649kPD46k/X/qLuKzhN49k50S3XJq5WwHi6CJvR8DzPOkjied0o1BHpfcCCIMorUcyfVxqmeu2m+uSE15ncQsd8BIvH4/YNoncUGYxqVhGB+FNUlWdhV0SPRW7V/gmyvFW5qdYX2QhNZAQHChdEH1E9AI7WiCFkyLRx39qYgXgqj8vOOnlwHgy/oJnpnu/Mlvj1uETX sage@montreal\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCVPfoCpv2/9uWCwEo4clWt3i1O8ZPpB2FcQbBLxY3iLQWZJDe74rNWPBj0cJVCytTHxYV0e5HxYyChOG60ca5odHlPNB+/Z4WLU9LVuHiCYsElivJlwvD+l9kzI+UMGJb3HCHqx2yIRGXpEkSMFR1+w5Oz9sInPD7JJmO8Hb9KkoeqZtkJL0aic/M5bDWBut6SSfYlmh8qnQYbkCDiXg/KtOcy1ZJXQ9ZLjsCbpSzeBNbdUOxEQ1+5vUe00vHpl0vOKtRrzgTne2ct1IxbSjdL16ZgHi5BQAREXc9UUG3z7NPOmnwkuKkkpyLmwcOusot5tYHrxks+5i6EoGEJrvJ3 sage@maetl", "key_options": null, "keyfile": "/home/sage/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sage", "validate_certs": true} 2020-04-10T06:59:18.102 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd', u'name': u'kdreyer'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4eHmz10szeHNS3dNejKokW85ksB+iR4HGOFsmQM11Ni68Nm5aqEKvkOZU8TpY92vpCQL0A68GlrXB845cACdyk6HUJYyNNNMC43l1FYWOwjMqQBSdj8W3VQDTA6eiG60mt5fgI8fyR38rKzIA1MnTBkSSjuh5kQVJ9bdEp3GuY5oc8vxDNBlGJ6LYnyEWt/pqL2J+mpjqnOjsC+EbE2exhP9O+mvzpQiyo/+dEN1COwX3//pNRXGfOSeOczHNsJE8Eu+j/n/BlW57++sJyFMkzS7bUxMSGM6quvjQZ7RT1c5JM6vLEiQyzQxoRgzY93h1yKlOstBi0NamtpqHQZGP kdreyer@mbp.ktdreyer.com", "key_options": null, "keyfile": "/home/kdreyer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kdreyer", "validate_certs": true} 2020-04-10T06:59:18.187 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n', u'name': u'dgalloway'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBAN3MlkYQNsm1uB6uJQ7hWJ1Ro9Gusrhi/b/KshPMFh51VWBQn71LdCCjdrsE6FWs5FYE/hr7MITJe+si7aZCmuf8U9tc+oiYh3spXtRhJVIM80FOVRXUmt2XR8I+oNbpcFDayYu956Volafksk3oYjsO356swj7+Q+AUQVF4bDJRAAAAFQCkB2YZKItSDL7XfAl6ZmJ21vqUMwAAAIEAsZspNIUwssVJGO4r3M3BECWklWLEFviIjGVKdPWulN10eRCczZrH7vMOs8coHKDQqYdS2RejlojT03FjFAqeBiTeAlcmgHoev+vqTg2Yhqx3gROzNc6NmG887owCOU4j7LJQP6SX8I8EpSXCMNOZliv2ZokjX2DuRsuIDxQdc84AAACBAMAp++fcqmGS7x7ic2x4rhyiYmdqlMIl8lrM3GkVR6qGuIVBvOGl38ieJKcTBLeRgyC04uIMfk8P9lLvZw3UF97qH3V0imKfq8aYnBGxCv+IjtdOrR5k4hrUW+P0qipvU+4dgHlpqwTKB4MtD9uDxvhENWvqcCnOS5rI6dddaZT6 dgallowa@dgallowa.csb\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZtt 2020-04-10T06:59:18.188 INFO:teuthology.task.ansible.out:Edz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/dgalloway/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dgalloway", "validate_certs": true} 2020-04-10T06:59:18.247 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n', u'name': u'dmick'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/onMoOCUhkp+UiJAq+ye4P70sfeV+uMhgWKLfLBh/1SYq1xxb9bSmjBnETO/9KvukRhzlReBQ7y+BKRhm037P7LuTUY4r0Y8JB6BXkjjCY1yXZJC6kvB7xFsvyiDum5rrdeWMQ/Rhh7YJpze5KY1nHLkyU9X7Ewak0kLEyduJEKekbsOom6l0wZ2ZYrBEKFWJXUDk5icUuWWg3mhKaKnLNWVg1lUw8jzYTJYfjR9GUgzZWXtzkAaI3ugpLOlQ+KEls03/WAFcWA9F9ExqzIsJpvEks27LeFBGVE3PjVNKMqqB 2020-04-10T06:59:18.248 INFO:teuthology.task.ansible.out:ZQlZtIEz1iguWJaWMi5lDIfXkvVOFabtIpX/BZRf dmick@dmick-MacBookAir\nssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOoIm6sOytKwqNf9FSQn0Olu72ZRDjAYeNFQVWTn2PIB ed25519-hplaptop", "key_options": null, "keyfile": "/home/dmick/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dmick", "validate_certs": true} 2020-04-10T06:59:18.322 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045', u'name': u'akraitma'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/akraitma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akraitma", "validate_certs": true} 2020-04-10T06:59:18.407 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n', u'name': u'sage'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJ4z0gLwjexgyixdTuiwaxpPnq7w4FOKxHsQKKAHTlKRu11ynB+9APbtqaAYss8yDqHaHyewPdE+9JMYhZX8rUgcKv2H7jBCQ/Enfl0UyY643VQnuG8lr2Ipk6ELuL5NuuTb7Npo4zKyq/0649kPD46k/X/qLuKzhN49k50S3XJq5WwHi6CJvR8DzPOkjied0o1BHpfcCCIMorUcyfVxqmeu2m+uSE15ncQsd8BIvH4/YNoncUGYxqVhGB+FNUlWdhV0SPRW7V/gmyvFW5qdYX2QhNZAQHChdEH1E9AI7WiCFkyLRx39qYgXgqj8vOOnlwHgy/oJnpnu/Mlvj1uETX sage@montreal\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCVPfoCpv2/9uWCwEo4clWt3i1O8ZPpB2FcQbBLxY3iLQWZJDe74rNWPBj0cJVCytTHxYV0e5HxYyChOG60ca5odHlPNB+/Z4WLU9LVuHiCYsElivJlwvD+l9kzI+UMGJb3HCHqx2yIRGXpEkSMFR1+w5Oz9sInPD7JJmO8Hb9KkoeqZtkJL0aic/M5bDWBut6SSfYlmh8qnQYbkCDiXg/KtOcy1ZJXQ9ZLjsCbpSzeBNbdUOxEQ1+5vUe00vHpl0vOKtRrzgTne2ct1IxbSjdL16ZgHi5BQAREXc9UUG3z7NPOmnwkuKkkpyLmwcOusot5tYHrxks+5i6EoGEJrvJ3 sage@maetl", "key_options": null, "keyfile": "/home/sage/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sage", "validate_certs": true} 2020-04-10T06:59:18.491 INFO:teuthology.task.ansible.out:changed: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9', u'name': u'jcaratzas'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcaratzas", "ovpn": "jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCgaz5AsLB+hPwnHjlf6vktlUQ8iWsKA+hViok+j+uCm0U/iho7O7Qd8E/3tp9yBx10Wkvuz1mmf6v2AO7Qa8+SE1QQPmA+4xLp4xXCqlUGFjnzm/7uQcQZtlWdfBHzQ9jVyPAgUpnGVUtW8PhW0txe8MEtecOO2C3Snx0yuTOx6O11jNvT938QjOeRQWw3X2YFUtMIOiwEhKtqstBIGXlMdaQKfvdlbA9MjyYy6VfK1x8lVcu8umUf/z56YmMkC2iO117/obhdu1XsU9SmjM51wHMOUCXSVqH8E3BWu0yohy4QmZDkdFihyYa+hoS9dOqGC91dFaQc46xrdmV4wqg1 jcaratza@MacBook-Pro", "key_options": null, "keyfile": "/home/jcaratzas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcaratzas", "validate_certs": true} 2020-04-10T06:59:18.558 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n', u'name': u'dgalloway'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBAN3MlkYQNsm1uB6uJQ7hWJ1Ro9Gusrhi/b/KshPMFh51VWBQn71LdCCjdrsE6FWs5FYE/hr7MITJe+si7aZCmuf8U9tc+oiYh3spXtRhJVIM80FOVRXUmt2XR8I+oNbpcFDayYu956Volafksk3oYjsO356swj7+Q+AUQVF4bDJRAAAAFQCkB2YZKItSDL7XfAl6ZmJ21vqUMwAAAIEAsZspNIUwssVJGO4r3M3BECWklWLEFviIjGVKdPWulN10eRCczZrH7vMOs8coHKDQqYdS2RejlojT03FjFAqeBiTeAlcmgHoev+vqTg2Yhqx3gROzNc6NmG887owCOU4j7LJQP6SX8I8EpSXCMNOZliv2ZokjX2DuRsuIDxQdc84AAACBAMAp++fcqmGS7x7ic2x4rhyiYmdqlMIl8lrM3GkVR6qGuIVBvOGl38ieJKcTBLeRgyC04uIMfk8P9lLvZw3UF97qH3V0imKfq8aYnBGxCv+IjtdOrR5k4hrUW+P0qipvU+4dgHlpqwTKB4MtD9uDxvhENWvqcCnOS5rI6dddaZT6 dgallowa@dgallowa.csb\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZtt 2020-04-10T06:59:18.560 INFO:teuthology.task.ansible.out:Edz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/dgalloway/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dgalloway", "validate_certs": true} 2020-04-10T06:59:18.623 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e', u'name': u'trociny'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDSc2rWtLtZM0CBWwaxhSfd0T++ZTX9c8kQrYT0rsQlJ6Hxj4c3+ypwh4iivdtFpyz4clxWkb/wimi48PvjNyZdudB9BeQUBeFfLiro3LnFEbKhM1RwRI89s34mRwmC0LLUZwcdX80zc6/fKuPzvBzR1PiY9tWRyIpYErQAHa2jEOE2MpFDHnlbXdfXFh5Tw3OmC1qBlQvgcrivA9lVaVlf+hfwmOQwXFIG3PCJLNlXDshCSUGAfYKBRvQaKS8sOf9rQaflPrwqJ8hDgVVQkpp69yVsGtahbBmjs6QS6C8EkGUFQmvrIcrQ8+5mRlRAmK2DMTP3cAcsYcMOJ+9KwP+H mgolub@zhuzha", "key_options": null, "keyfile": "/home/trociny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "trociny", "validate_certs": true} 2020-04-10T06:59:18.653 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007', u'name': u'smithfarm', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:18.706 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045', u'name': u'akraitma'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/akraitma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akraitma", "validate_certs": true} 2020-04-10T06:59:18.787 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n', u'name': u'mbenjamin'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANarZ+C62DPffRIhaCuCtXLChrfqwXgWqGHOLLkmPTtZ3/EE8ZiN+LY1ZcmP939AxXi8Oe+9LepwAdsoQKZctlC7XToyhjIk+rVSPyn3zl3BTC2AxfCKq2vqj65a1sc35Whrw7OaUa2fr5iHSazP6HuG01DhJ9l8HJsLKmiwWChZAAAAFQCNmunc7BNLWrqyrYwINTfscowkbQAAAIEAyYERcunZYJAZ9Eno18kM+ylq0jcQmnPqyIvssWXTttUZnaGQvbVxABeyr6ZKuUyZQfyetofquOGBKPBuzV/7posl5ShDuAcxbS1zhrmk9glTA1yFLvznDdWdLOWG06tjOc78G2AmldKHCdPsmgh2D7WyRr4QY4rcCvwKfZ6+eEYAAACBAJZrOlOlygfPZ4ZB0+EbdYACz1+L/iYNNAOBp51TRutNceLVrNJNOVdmCMLuF0A1go4kPSD/05aJOkOyW86gYLA5DXaW3961+kPInOFAe8OTyQYE7221ThYHRnCfVFjdknuCDomj69wk4SsR+ZPXvSXs1jSJWKhLfUSe9n4bmw7w anet2@secant.private.linuxbox.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5CMht/crEocrHqGjf0sxE9AhNcW3XBxwlniJwaBJLhekLQEQJoxAy2xUUmPYb2/e7rdmTBxzQipUaAGWVk1FHANac4cbgXzNT+9cnF7EIRkPa/x9jaKNiU0ozMhu/GOoeDXMrHIDgVGWPF3Ag/r17ORpYnuFTjsqoCWyh+mqskV+lmAp04rv2tvqouHtEs6gLebKNtB8c6twhrBKF62Ie+rSr3r7hQaqiZitHbg/isbU5RMBl6H48wHAVSK2/uT 2020-04-10T06:59:18.789 INFO:teuthology.task.ansible.out:fMorbnuEYRG8HvlnPRvQHscJYKiZSt/G7XqZ3x/7c4czEF+OpdQbAgg46vh4fNVMxZk9KHwEkOE06ROuwgAejJ mbenjamin@duel", "key_options": null, "keyfile": "/home/mbenjamin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mbenjamin", "validate_certs": true} 2020-04-10T06:59:18.863 INFO:teuthology.task.ansible.out:changed: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9', u'name': u'jcaratzas'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcaratzas", "ovpn": "jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCgaz5AsLB+hPwnHjlf6vktlUQ8iWsKA+hViok+j+uCm0U/iho7O7Qd8E/3tp9yBx10Wkvuz1mmf6v2AO7Qa8+SE1QQPmA+4xLp4xXCqlUGFjnzm/7uQcQZtlWdfBHzQ9jVyPAgUpnGVUtW8PhW0txe8MEtecOO2C3Snx0yuTOx6O11jNvT938QjOeRQWw3X2YFUtMIOiwEhKtqstBIGXlMdaQKfvdlbA9MjyYy6VfK1x8lVcu8umUf/z56YmMkC2iO117/obhdu1XsU9SmjM51wHMOUCXSVqH8E3BWu0yohy4QmZDkdFihyYa+hoS9dOqGC91dFaQc46xrdmV4wqg1 jcaratza@MacBook-Pro", "key_options": null, "keyfile": "/home/jcaratzas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcaratzas", "validate_certs": true} 2020-04-10T06:59:18.931 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01', u'name': u'aemerson', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:18.942 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78', u'name': u'amaredia', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:18.953 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5', u'name': u'tserlin', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:19.027 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e', u'name': u'trociny'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDSc2rWtLtZM0CBWwaxhSfd0T++ZTX9c8kQrYT0rsQlJ6Hxj4c3+ypwh4iivdtFpyz4clxWkb/wimi48PvjNyZdudB9BeQUBeFfLiro3LnFEbKhM1RwRI89s34mRwmC0LLUZwcdX80zc6/fKuPzvBzR1PiY9tWRyIpYErQAHa2jEOE2MpFDHnlbXdfXFh5Tw3OmC1qBlQvgcrivA9lVaVlf+hfwmOQwXFIG3PCJLNlXDshCSUGAfYKBRvQaKS8sOf9rQaflPrwqJ8hDgVVQkpp69yVsGtahbBmjs6QS6C8EkGUFQmvrIcrQ8+5mRlRAmK2DMTP3cAcsYcMOJ+9KwP+H mgolub@zhuzha", "key_options": null, "keyfile": "/home/trociny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "trociny", "validate_certs": true} 2020-04-10T06:59:19.080 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28', u'name': u'dis'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHk8c8wUTTYolOOmMlniNrsjfRyF9pCCnaEQ2DXR73P0hbFMme5Md5cF3PTtPIOHSNkWk/qg7s6gZD8LGyFmloO42RBgLsW1qM8RWTDbPy5EQdrjxL9byVzBBCDoQPIS2M+0XnhYa6CBj38Sw8XPlS41CebAJEOQeyLG2I6CAI0rTij6gsqDfALbIezd/eWvmbksKYM1Lb4Gvj+k/OL/xDZzIZHF/P+mK+trY45gsUHxKZamkeHyAT4f3kiNJA+UwzTWGhafy6UE7yGqhZjt9xOZSQGuwroL7EtAzntxayPRNxQPfGDV+fYYf0QBbTtHyJeikPUuw7aR1Z3KwuSmWD dis@zambezi", "key_options": null, "keyfile": "/home/dis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dis", "validate_certs": true} 2020-04-10T06:59:19.166 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007', u'name': u'smithfarm', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:19.229 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n', u'name': u'gregf'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANaS5gZUnIp5UL/PM3xOrQiNVxfona7sD1XEXGYvNUyoojPWWAvdQRwdA/Ay8JsAjKeWUt0IwfWaPfbEoXU1TKAI+yyv5WzQUZAKmBFuYM0CUQnKrk72Dp6n8P2tnYen0ect22xEiW2+Y0JZ3wfV3IB3BQPoVCC9FQ+P3LHhYii/AAAAFQCgSceiZFeNItwJ2rJeQgVVTQKljwAAAIEA0Gd2VNCbbQ48vAF+m/i/CeQHJ9Dom9IOBN3v8XnSlOhtNzY1pJ0NRzrFQURCCeVQJaywr0et5FCXIDVqTuR4qfCu5XXU1KHpn/AvHstOd93Y0XYMwSXfNszOtpH7w9SEm7uBfKnLQNF843/MHZL4ZIdfxk73YjfMjrYFcvJwvM0AAACBAI9Wi2wG9Rrdbrgk8tqbnKMxxJCYt/Vakn5mQSc/WHqMUn14+Kb7oBtfTsfPFNK952FzU09ptNQvNWTPNMniI8l8lQi2B+Z28XUlFLL76uxJXWko99xFpf/d1kFBL0M7GRGmvvrMfeTCI+WIqh2jBVqbl4o8kjNx4p0VWyoiWeFJ gregf@pudgy\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAwSe7szyqMTqKK3D/JsXIa1PtPf1p8Njk0sMDSsJPA0gBGysmSgstke9fglOgeNIbNOBNDOY3vltuNCgUiQgVWOYhUaVpj+ 2020-04-10T06:59:19.231 INFO:teuthology.task.ansible.out:2vtSoAWpXMQbNaWPGKhbxvJWMCLVckLENyoUZUBKLmTvRyKDjjq8CJ1TnfA2Xfo3a796EIOtD6essU3lAVIfIUILGD+HqptiHSB0spUJUVRxxlO2xw9rLYij1eUPRHrr9xWvDamr5JtKVtCLOL3M9QlltHieKlbJSCZq/EVxW9NAGA6IiqOhw2E1k59oGZUq4Qg+YLgYkB80Ms7nb1xpjQ8txvY+73xL/8i7Nqmr7ekKJfyTpRe03nCQ== gfarnum@mozart.thompsonian.net\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzqJQfdAKaggPDgSpk8P6QjLeyQYOz183fVSKnqAa5EeaCJozS1dn8z+iphswT3YL6wqIMqVHMfkSUcEeTBwZmvydaghOMJEIQSP1PCzH0V2XprELrEcBC3QTvaElPjjEYgBVUvEw8sL/RIQ7mgt9vxD0NcCDspsz35fmU222KzFxD41PZ5XzHQ3Lumwdax7mcpqrw8Aa5/tgcEqAIu54HYRf1Qmk6+ueMAvpwkX3eiSEtkVvuV1Kxpc3/0mOfJ7cn+eyPX3qUbo4oY76hIYseoUIwEaVE/pNW+pjwDSy4gSOcuxYDEGKzCwZPZeQtRQWzRt3YNafxi5xsByobGLlZw== gfarnum@GF-Macbook.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1ik3LbyKsPiGwdh+vGrKSYLvhMg7OkqMwDNYU5LqWBSotogxyGQJUuD80ZavxmmdXC4qScddHtWOJjtyX1CTSDUe7INpSgGPGnmSExVFpcYRnmxHAB+9M/c9VorUkQ8IOKLmRO/BdYma3Rl9NzhHI73yd4zJlx1K4NuQnhnj0Z4yXO+k44RkBWDdMicBB5lBP0hETaH2Iq1u5Dai3u6N5N9bSPXv2PGKvvU395K0Wts2CxMhOBc7LAixysNzNWLrnG1ghS4jPVG2TIHOElP5lIXUv1Wlkool5rTUWgnVY/7loWkDus2JryDM7hLGFXqgT7HHcpI4gSITqRhWxzsUF gregf@fedoragreg.localdomain", "key_options": null, "keyfile": "/home/gregf/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gregf", "validate_certs": true} 2020-04-10T06:59:19.306 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n', u'name': u'mbenjamin'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANarZ+C62DPffRIhaCuCtXLChrfqwXgWqGHOLLkmPTtZ3/EE8ZiN+LY1ZcmP939AxXi8Oe+9LepwAdsoQKZctlC7XToyhjIk+rVSPyn3zl3BTC2AxfCKq2vqj65a1sc35Whrw7OaUa2fr5iHSazP6HuG01DhJ9l8HJsLKmiwWChZAAAAFQCNmunc7BNLWrqyrYwINTfscowkbQAAAIEAyYERcunZYJAZ9Eno18kM+ylq0jcQmnPqyIvssWXTttUZnaGQvbVxABeyr6ZKuUyZQfyetofquOGBKPBuzV/7posl5ShDuAcxbS1zhrmk9glTA1yFLvznDdWdLOWG06tjOc78G2AmldKHCdPsmgh2D7WyRr4QY4rcCvwKfZ6+eEYAAACBAJZrOlOlygfPZ4ZB0+EbdYACz1+L/iYNNAOBp51TRutNceLVrNJNOVdmCMLuF0A1go4kPSD/05aJOkOyW86gYLA5DXaW3961+kPInOFAe8OTyQYE7221ThYHRnCfVFjdknuCDomj69wk4SsR+ZPXvSXs1jSJWKhLfUSe9n4bmw7w anet2@secant.private.linuxbox.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5CMht/crEocrHqGjf0sxE9AhNcW3XBxwlniJwaBJLhekLQEQJoxAy2xUUmPYb2/e7rdmTBxzQipUaAGWVk1FHANac4cbgXzNT+9cnF7EIRkPa/x9jaKNiU0ozMhu/GOoeDXMrHIDgVGWPF3Ag/r17ORpYnuFTjsqoCWyh+mqskV+lmAp04rv2tvqouHtEs6gLebKNtB8c6twhrBKF62Ie+rSr3r7hQaqiZitHbg/isbU5RMBl6H48wHAVSK2/uT 2020-04-10T06:59:19.308 INFO:teuthology.task.ansible.out:fMorbnuEYRG8HvlnPRvQHscJYKiZSt/G7XqZ3x/7c4czEF+OpdQbAgg46vh4fNVMxZk9KHwEkOE06ROuwgAejJ mbenjamin@duel", "key_options": null, "keyfile": "/home/mbenjamin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mbenjamin", "validate_certs": true} 2020-04-10T06:59:19.318 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01', u'name': u'aemerson', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:19.330 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78', u'name': u'amaredia', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:19.345 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5', u'name': u'tserlin', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:19.378 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n', u'name': u'joshd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDgS5EfJyOxBkl/TtJikfb38rm6muQo7MwYufkLSBw9W2jFzt/ZMbVXyVsdZM4I6hWfZm1/q60UcER4yoA1xwYGOTmbFXrLUqqKxKTnCdyejuQ3miB7aiuisWO4vjb2+ppBwL9Mx775DPeQJehFyFK0qgh33tZO/9LWWYy7NOZdA9L2ei1PgMWa0VBeijcDR5GQSXzh0Yi+h/Ezu8jevu+NkW2IUJF6zPGjd9M9lAy7KG2dQjqqCX4XWmUS+MHJS6f9bsC5i39GWy29FvX/qFEArOMzuYRIFWE8iza95KSBxptb+Hw8PFBjQ1kTh6nC5ZV2X3ETcoj8fToTcgW3dTRF jdurgin@new-angeles", "key_options": null, "keyfile": "/home/joshd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joshd", "validate_certs": true} 2020-04-10T06:59:19.485 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28', u'name': u'dis'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHk8c8wUTTYolOOmMlniNrsjfRyF9pCCnaEQ2DXR73P0hbFMme5Md5cF3PTtPIOHSNkWk/qg7s6gZD8LGyFmloO42RBgLsW1qM8RWTDbPy5EQdrjxL9byVzBBCDoQPIS2M+0XnhYa6CBj38Sw8XPlS41CebAJEOQeyLG2I6CAI0rTij6gsqDfALbIezd/eWvmbksKYM1Lb4Gvj+k/OL/xDZzIZHF/P+mK+trY45gsUHxKZamkeHyAT4f3kiNJA+UwzTWGhafy6UE7yGqhZjt9xOZSQGuwroL7EtAzntxayPRNxQPfGDV+fYYf0QBbTtHyJeikPUuw7aR1Z3KwuSmWD dis@zambezi", "key_options": null, "keyfile": "/home/dis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dis", "validate_certs": true} 2020-04-10T06:59:19.517 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n', u'name': u'davidz'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCyOWnAdvzh5zR2ts9e8g0wG4wZ0uNr0YJUCncAOTJB0L8xkQU76jWzX3BRNU5Wk8VtLvpJAQ61P7UXyzgGWSO9FHZjD3RDrc2V9uh/lqRKmx1COtDvcart/JwzwHlsWdrYdcXvEaAiFrZJ6x1jJAjIL6tvMO7bqBz4YNruKpGWsQ6p5P/M5iIEPMj1oKanLoULXqReC6eYFTw3LNAXjberPY5GxYwaKS0HuzySRvSblShyA85jawTSlEbsOSnMqqGu5vGJiC415UFQQ5qKBtzrgAjlUgMbaqGTnTmBYn7L751gIWRetgWqwuHgfIe2ZGM3mmh0DX8E03DUbEt4UxyR dzafman@TrustyTahr\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDDtSwGzH7V5jzppggL0+nYlBSM1HqGjkInwxutra5iEyFoBuV73MNSBk3HrvIVGN13+PCdHf4Oy8ywbhC/4yKbxS1NG76d/GU8ijZFuBlEwxwHZpcThpSTuYrNx7FJVp8Yjl45DtD5s8FMI/0YZZhLlIqT6QCTxlTIPXDb6zefEWvAVziE0bmhRpe4B6AOLOGTB9fivQr5YzxhfaSS5Vu4U1x5gmosouSJ+aMFrQWLqa/eoCy8t9PnxdMjlHjHCZ7Qv1JfpaAPw3qOK30GrVeXR/hPN3aFodTuIJpzdynjk5HO6/VKW5+mWBO7pmQqn9LNQEjADXbLh7PG//nOONbH dzafman@ubuntu16", "key_options": null, "keyfile": "/home/davidz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "davidz", "validate_certs": true} 2020-04-10T06:59:19.636 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n', u'name': u'gregf'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANaS5gZUnIp5UL/PM3xOrQiNVxfona7sD1XEXGYvNUyoojPWWAvdQRwdA/Ay8JsAjKeWUt0IwfWaPfbEoXU1TKAI+yyv5WzQUZAKmBFuYM0CUQnKrk72Dp6n8P2tnYen0ect22xEiW2+Y0JZ3wfV3IB3BQPoVCC9FQ+P3LHhYii/AAAAFQCgSceiZFeNItwJ2rJeQgVVTQKljwAAAIEA0Gd2VNCbbQ48vAF+m/i/CeQHJ9Dom9IOBN3v8XnSlOhtNzY1pJ0NRzrFQURCCeVQJaywr0et5FCXIDVqTuR4qfCu5XXU1KHpn/AvHstOd93Y0XYMwSXfNszOtpH7w9SEm7uBfKnLQNF843/MHZL4ZIdfxk73YjfMjrYFcvJwvM0AAACBAI9Wi2wG9Rrdbrgk8tqbnKMxxJCYt/Vakn5mQSc/WHqMUn14+Kb7oBtfTsfPFNK952FzU09ptNQvNWTPNMniI8l8lQi2B+Z28XUlFLL76uxJXWko99xFpf/d1kFBL0M7GRGmvvrMfeTCI+WIqh2jBVqbl4o8kjNx4p0VWyoiWeFJ gregf@pudgy\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAwSe7szyqMTqKK3D/JsXIa1PtPf1p8Njk0sMDSsJPA0gBGysmSgstke9fglOgeNIbNOBNDOY3vltuNCgUiQgVWOYhUaVpj+ 2020-04-10T06:59:19.637 INFO:teuthology.task.ansible.out:2vtSoAWpXMQbNaWPGKhbxvJWMCLVckLENyoUZUBKLmTvRyKDjjq8CJ1TnfA2Xfo3a796EIOtD6essU3lAVIfIUILGD+HqptiHSB0spUJUVRxxlO2xw9rLYij1eUPRHrr9xWvDamr5JtKVtCLOL3M9QlltHieKlbJSCZq/EVxW9NAGA6IiqOhw2E1k59oGZUq4Qg+YLgYkB80Ms7nb1xpjQ8txvY+73xL/8i7Nqmr7ekKJfyTpRe03nCQ== gfarnum@mozart.thompsonian.net\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzqJQfdAKaggPDgSpk8P6QjLeyQYOz183fVSKnqAa5EeaCJozS1dn8z+iphswT3YL6wqIMqVHMfkSUcEeTBwZmvydaghOMJEIQSP1PCzH0V2XprELrEcBC3QTvaElPjjEYgBVUvEw8sL/RIQ7mgt9vxD0NcCDspsz35fmU222KzFxD41PZ5XzHQ3Lumwdax7mcpqrw8Aa5/tgcEqAIu54HYRf1Qmk6+ueMAvpwkX3eiSEtkVvuV1Kxpc3/0mOfJ7cn+eyPX3qUbo4oY76hIYseoUIwEaVE/pNW+pjwDSy4gSOcuxYDEGKzCwZPZeQtRQWzRt3YNafxi5xsByobGLlZw== gfarnum@GF-Macbook.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1ik3LbyKsPiGwdh+vGrKSYLvhMg7OkqMwDNYU5LqWBSotogxyGQJUuD80ZavxmmdXC4qScddHtWOJjtyX1CTSDUe7INpSgGPGnmSExVFpcYRnmxHAB+9M/c9VorUkQ8IOKLmRO/BdYma3Rl9NzhHI73yd4zJlx1K4NuQnhnj0Z4yXO+k44RkBWDdMicBB5lBP0hETaH2Iq1u5Dai3u6N5N9bSPXv2PGKvvU395K0Wts2CxMhOBc7LAixysNzNWLrnG1ghS4jPVG2TIHOElP5lIXUv1Wlkool5rTUWgnVY/7loWkDus2JryDM7hLGFXqgT7HHcpI4gSITqRhWxzsUF gregf@fedoragreg.localdomain", "key_options": null, "keyfile": "/home/gregf/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gregf", "validate_certs": true} 2020-04-10T06:59:19.648 INFO:teuthology.task.ansible.out:changed: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed', u'name': u'cmeno'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "key": "ecdsa-sha2-nistp521 AAAAE2VjZHNhLXNoYTItbmlzdHA1MjEAAAAIbmlzdHA1MjEAAACFBAHT7CHaceRLB5mJv9SYKog1wBXsavzH3G3WCBEvVyK76JIslDudZm2ycJv66r+VlGaD8sFI5Mb/KKFMKcKesLRX+wHqZg9MlkoJgm7WGoRQ5I0M1Vt7TL3uJkVCaGZHtO7BXqzTx53C1AaJ1HN+DG2MGy5mm6tHwL901UVqG4EzjBnjxA== christinameno@boxtop", "key_options": null, "keyfile": "/home/cmeno/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cmeno", "validate_certs": true} 2020-04-10T06:59:19.775 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1', u'name': u'ivancich'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDkIZyWnJXYv/z3EVeKHatjFUAdaomIMqzvFLrA9VskeM0Ctv8a83Wk3BbYvTFQWALjZjS2HVYzi5p6kTQ4qxXlcO7HxRw0j4otNauilvgIc4Lu5N4pUtYf0piEo+nrXz+WhELfmJ8t1Yx00uGEl9pOF04SOuXtpwwrCFAO6WMCcjDuYhoI2VBKO3W/OWcIuQ8ZY3j4iDdofFOUGyyoYVyOi/HILKjluYB+ues5+wBEkDuoi3MafCnIj8M0O5fU8/JoOTksOu779UN6blPiNdEgnwD00jA+RWYY6b12ZVkiF4rUVGCiiKKUwOOWkRhkA5OEcBgZ8cGWFgGfZM1GJA// ivancich@sleepy", "key_options": null, "keyfile": "/home/ivancich/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ivancich", "validate_certs": true} 2020-04-10T06:59:19.792 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n', u'name': u'joshd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDgS5EfJyOxBkl/TtJikfb38rm6muQo7MwYufkLSBw9W2jFzt/ZMbVXyVsdZM4I6hWfZm1/q60UcER4yoA1xwYGOTmbFXrLUqqKxKTnCdyejuQ3miB7aiuisWO4vjb2+ppBwL9Mx775DPeQJehFyFK0qgh33tZO/9LWWYy7NOZdA9L2ei1PgMWa0VBeijcDR5GQSXzh0Yi+h/Ezu8jevu+NkW2IUJF6zPGjd9M9lAy7KG2dQjqqCX4XWmUS+MHJS6f9bsC5i39GWy29FvX/qFEArOMzuYRIFWE8iza95KSBxptb+Hw8PFBjQ1kTh6nC5ZV2X3ETcoj8fToTcgW3dTRF jdurgin@new-angeles", "key_options": null, "keyfile": "/home/joshd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joshd", "validate_certs": true} 2020-04-10T06:59:19.936 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n', u'name': u'davidz'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCyOWnAdvzh5zR2ts9e8g0wG4wZ0uNr0YJUCncAOTJB0L8xkQU76jWzX3BRNU5Wk8VtLvpJAQ61P7UXyzgGWSO9FHZjD3RDrc2V9uh/lqRKmx1COtDvcart/JwzwHlsWdrYdcXvEaAiFrZJ6x1jJAjIL6tvMO7bqBz4YNruKpGWsQ6p5P/M5iIEPMj1oKanLoULXqReC6eYFTw3LNAXjberPY5GxYwaKS0HuzySRvSblShyA85jawTSlEbsOSnMqqGu5vGJiC415UFQQ5qKBtzrgAjlUgMbaqGTnTmBYn7L751gIWRetgWqwuHgfIe2ZGM3mmh0DX8E03DUbEt4UxyR dzafman@TrustyTahr\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDDtSwGzH7V5jzppggL0+nYlBSM1HqGjkInwxutra5iEyFoBuV73MNSBk3HrvIVGN13+PCdHf4Oy8ywbhC/4yKbxS1NG76d/GU8ijZFuBlEwxwHZpcThpSTuYrNx7FJVp8Yjl45DtD5s8FMI/0YZZhLlIqT6QCTxlTIPXDb6zefEWvAVziE0bmhRpe4B6AOLOGTB9fivQr5YzxhfaSS5Vu4U1x5gmosouSJ+aMFrQWLqa/eoCy8t9PnxdMjlHjHCZ7Qv1JfpaAPw3qOK30GrVeXR/hPN3aFodTuIJpzdynjk5HO6/VKW5+mWBO7pmQqn9LNQEjADXbLh7PG//nOONbH dzafman@ubuntu16", "key_options": null, "keyfile": "/home/davidz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "davidz", "validate_certs": true} 2020-04-10T06:59:19.950 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n', u'name': u'wusui'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsTZI+b02boZv8jVLHQoFlloD2WF/3xT+ZfZxzwddL7Q6eqo7j8i21LAbLAMWXYMyhiClrloDeHfCZ42gG5bqSThF8LoCw+GspF6SahTI0lmptGu69/OeHqqCpW1DnsWB3wWqfi90XbQJrpg3cdRaqOCmkGVJrqfFWzff8Zg9IuojIH/aO9drCTRGa1pHHCyEdW9ZNeYFwrMjfP+wJi/mC/ABAoXyDy0NWT0uIA+36abNg2/ptok90rdan0r72hjFEz/1hpupKWri+VXAepiyDMuSgJm9uJCXDFaNTn3jocdX+sbVtltThQ2OBxe1T60vRLbXx3lvhxxZ1fUkcP/FF wusui@ubuntu\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD2UzyfBk1nZ8SLCtW9UjorMeOo4u9AM/YL8ZI74GdprR+odERZR17/QfwT5vmLVNa51+CQt1HcGodlQYfR/8n/OUl/gJyVagBuU2zgmLPO3tOor5RBfms5MZ2VZ9Vm3+WrCO/vrAq/5HVyGbQsKiQHXkIRSL2NIgfKRTOVTNmuTSXtZNDgdWGIBmcOFQv+Im9CRopoI8ZtJLQj7xfxtGtUas7FbWTcf0o92+xvWuGWq5WQgmZ8MUIfvBLYLmicuVfsWjZ1YsC1PNhuhkwrW7MhaD71Ej3ll243tG1QD7eAmo+psIp8phlU5s/5FeA5y5OzDZ2T963oHGhq018TYO0b wusui@teuthology", "key_options": null, "keyfile": "/home/wusui/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "wusui", "validate_certs": true} 2020-04-10T06:59:20.071 INFO:teuthology.task.ansible.out:changed: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed', u'name': u'cmeno'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "key": "ecdsa-sha2-nistp521 AAAAE2VjZHNhLXNoYTItbmlzdHA1MjEAAAAIbmlzdHA1MjEAAACFBAHT7CHaceRLB5mJv9SYKog1wBXsavzH3G3WCBEvVyK76JIslDudZm2ycJv66r+VlGaD8sFI5Mb/KKFMKcKesLRX+wHqZg9MlkoJgm7WGoRQ5I0M1Vt7TL3uJkVCaGZHtO7BXqzTx53C1AaJ1HN+DG2MGy5mm6tHwL901UVqG4EzjBnjxA== christinameno@boxtop", "key_options": null, "keyfile": "/home/cmeno/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cmeno", "validate_certs": true} 2020-04-10T06:59:20.086 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304', u'name': u'zyan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCtmxLK2rSomB14d9/fkAshmsXQfd6cJ+vEb0MZDaUT+OLmr1vAq0+H5g9w3hApJ2UEOFSaiz6ywgBAFeBeu/KmUALOhJIKKk0imoTunYclqxVPBK6tG/2b+ME8VE1afoEpMem3Bzfkrq0pDt6DLFBDlt9rRIqtO/ZlqEayWkEwTXJ7pwj+oJS03Z3Y7q0Iqw8PToKqJWwy2OvW1I9Pi0Q8s4YV9I34rSO7Wv4w43AXpFHRlvJWIcI5g5WFI5tnjel+kUAWMZft9ynFoAP5iXdByq897r4jAF74hapaxLMsHBz4MyYmPZqS76Cx2e3kGqXeetPinfcZxC3ViboR7o/1 root@zhyan-desk1", "key_options": null, "keyfile": "/home/zyan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zyan", "validate_certs": true} 2020-04-10T06:59:20.227 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n', u'name': u'yuriw'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCw3MhMNDYjkiQADoC1mg9ga0Yj174FHIFt4RQIC8qK+6cB0N3CM5R4JwP8pR4tYkVFWmnYRScYK5VVdC873Ac2BzYxmse97zLJTbUnYWTdY+OFi1FUozsy3PwtiNHSd48Z/PBvkinQIey2QlMKaW3/yz9Of/Yfm4prxg9lBsd6/tOOFj3AsqIQ1OVhMfgeVEkdCL7AAbEYyMxePZ3UFrDwj05HDLgmFzx1xsNJFOuBwLfU/QNHoXJQrcVy0rZq9fB5rg74iz1JZs0uDWocQQK4Nd3Rv+wRGBuew2fmkrS2bQWXzMoSL+/i7pbf74LKcbWAE0wXEImqQeREHxq4ZZ5J yuriweinstein@Yuris-MacBook-Pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJUmq7btpBMjjODHXp1888xoBuX+rhiZg3ufuo69RgDxUMymJn3CPJ/5NGXoZ3SGAWKk0w0Dx6K/VO4y92mAV+cO81y8wZR4J3fPV3IbWRjN0X3IFoAvJi5n9v5hpG7z3RZCGmQnOZ99m1i9dge/3S8NpuPLIvYkSo9z00d+mNP6D8rGnqOmEBfEhUQXLeH7rbPwaMWPgB/hrpiPCyLR/+sGC5bq9PAsKnocqZ12cnvCtYxpnA7et6nL15ftEpUrKYAmRDxotuoUcsllP66GSszjbrgN6OBNDSqpGt/+RSJJwzZEGQ073gJuMveKy9tvD/PyEttjvVKL2KsBuXGSuL ubuntu@yw\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmXWgRRes8bzekTH2fTcxdAAquFn1SjPhw 2020-04-10T06:59:20.229 INFO:teuthology.task.ansible.out:s1Z9CkxNaqKe25XnzPehqrhg9etouUwvORQHdDJdTlmhTSYtn588Omq+1bvniab0KzAYgGZsWdPma/3E9qF7ChXIMik9Umip0drzYTaexCuC/Nu46brTxlnL0vaxnnVjDwmHC0CEJs4qN4e2wSlsA0uBDRWjxWFMdTUgSzII5xasHypUYZ0y3MNdbfY+SVF69DEDH8EOQMGYm8JQXNJFloQvmvLUFqT/63FsdpzRVTph4A8hvI1f8i6ULb7fPQN5ZS62wdXd/KvrBXrCvhOi7R63RM60mkMRXm1XajTGmCKiikfiqgTr yuriw@Yuris-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC41kRr6B/RC32DrQKRByalH832H6BpCQF9XdTvgYi4+LJt+kAuIGLDF4xm4uApTmWRrLspWm0Kksc0ger03uNx5qJqMEUnJvjxJnSDC0/XyxWapVxIviFkvv0BjbhVNOOaMU55ywrZ9hHgMdaFwuMuCOaB7f9Tm6IUXm2k82OXueeIG23Kziq+IQBysWOjvNUy/LLpl7l4+xOkhNUupUiQCqHvFH98VUdYWkHeyamafGVZmYbFLrOXgmviKo0u8LPxcT7psKLk0yFG7fG0DSak6DDdKJqb+G/NqMSPxRSAI4FbXQ6UOZb32Ozr1Sql9yfOXEYWZuatn7W5F8DjSv3p yuriw@yuriw-RH", "key_options": null, "keyfile": "/home/yuriw/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuriw", "validate_certs": true} ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1', u'name': u'ivancich'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDkIZyWnJXYv/z3EVeKHatjFUAdaomIMqzvFLrA9VskeM0Ctv8a83Wk3BbYvTFQWALjZjS2HVYzi5p6kTQ4qxXlcO7HxRw0j4otNauilvgIc4Lu5N4pUtYf0piEo+nrXz+WhELfmJ8t1Yx00uGEl9pOF04SOuXtpwwrCFAO6WMCcjDuYhoI2VBKO3W/OWcIuQ8ZY3j4iDdofFOUGyyoYVyOi/HILKjluYB+ues5+wBEkDuoi3MafCnIj8M0O5fU8/JoOTksOu779UN6blPiNdEgnwD00jA+RWYY6b12ZVkiF4rUVGCiiKKUwOOWkRhkA5OEcBgZ8cGWFgGfZM1GJA// ivancich@sleepy", "key_options": null, "keyfile": "/home/ivancich/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ivancich", "validate_certs": true} 2020-04-10T06:59:20.361 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n', u'name': u'wusui'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsTZI+b02boZv8jVLHQoFlloD2WF/3xT+ZfZxzwddL7Q6eqo7j8i21LAbLAMWXYMyhiClrloDeHfCZ42gG5bqSThF8LoCw+GspF6SahTI0lmptGu69/OeHqqCpW1DnsWB3wWqfi90XbQJrpg3cdRaqOCmkGVJrqfFWzff8Zg9IuojIH/aO9drCTRGa1pHHCyEdW9ZNeYFwrMjfP+wJi/mC/ABAoXyDy0NWT0uIA+36abNg2/ptok90rdan0r72hjFEz/1hpupKWri+VXAepiyDMuSgJm9uJCXDFaNTn3jocdX+sbVtltThQ2OBxe1T60vRLbXx3lvhxxZ1fUkcP/FF wusui@ubuntu\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD2UzyfBk1nZ8SLCtW9UjorMeOo4u9AM/YL8ZI74GdprR+odERZR17/QfwT5vmLVNa51+CQt1HcGodlQYfR/8n/OUl/gJyVagBuU2zgmLPO3tOor5RBfms5MZ2VZ9Vm3+WrCO/vrAq/5HVyGbQsKiQHXkIRSL2NIgfKRTOVTNmuTSXtZNDgdWGIBmcOFQv+Im9CRopoI8ZtJLQj7xfxtGtUas7FbWTcf0o92+xvWuGWq5WQgmZ8MUIfvBLYLmicuVfsWjZ1YsC1PNhuhkwrW7MhaD71Ej3ll243tG1QD7eAmo+psIp8phlU5s/5FeA5y5OzDZ2T963oHGhq018TYO0b wusui@teuthology", "key_options": null, "keyfile": "/home/wusui/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "wusui", "validate_certs": true} ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n', u'name': u'tamil'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": f 2020-04-10T06:59:20.363 INFO:teuthology.task.ansible.out:alse, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxrS30JmTZbnBd73s9PGZYxCkWH3u4En9Q0bYJan45W+j+y5vMzaNUVsp8We1tphqPQ9B0ygi80OxbHWu5gyv4g+M/JFtrnxv3QgBCYWeBWfh652Vbdlk1thl6btFnHUhVMSn5tkdnFxyVDJHIROoVXkHRaCNAUUy7tB60IhPAMomNcCGu9R8+opONFLXKTIfkYQ1cGbysr90DMs8ASEkvxdv40hIWhK4ufvL7CFFtgXuoC0c1WedBFOkr5LDegzL13xN97K9o5J8CAU0MUKk6H8KNKbQuNq1zANwymMFY599GIlxwai8iluJgfOObDRdCDmKM2f9RAEUXQM7oYoH7 tamil@tamil-VirtualBox\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvwiWVhNLeOLg72cpVIwbUYnmFh+Cqf/mE7Rc/Xci91on3GPrq++dC8t2GhTB5Zyq46EE27SmUkiZpHRnz7xwiJKM/tTgNEkffXDK3j0pA2E22iqV9FpfSA9Zdi1Puq7CPT7/rix4rF3SPvMtg/R1fkyornLFdzhJiMExCKJXGwf+QsKk4NVnr+nO3EEkKOoP1wio1bceZ53rCt9Tyu8NawAB2nL2fvtBPQDAriQ8bLX69XLbOfhFI6GJo815/j6CQxNmvA0l5ABJc8+A71jZmndzumFRgqBuhiCM5ohsGR0JGahUQjFEA4quNTx8EWTGRsqhAYszzpqJzuN+Xb6tJ tamil@macbookair\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEPzEyDGI7poyXVPG70vgQpvlU/lTt6Vvv5CoHWJoSrHklDdJEqekhCB87BX4DTrqbb5xoM7yfxle/z9UzgVwSUWqsPdZHA4EbrNFRKFxi/gc2YrygVSjumTyzf9WJJ5vCZvTQexOj7rjWVnJkjXHKMXZXIAQN/bJf6J21t+4A3XLrl30/4XXyq664NONUNjBHxHtlbS2LM6Th/hK4B23oj2BugX2j/50vWgd9H36bPr8+JGskOi6MH2ndZv5pYtLznFzQmsbixjaN8kCuyQdlRCOndPEFoHFhDla2MeB+h3XQLDXW7J1TR2URFSfcNq5ndDS4i0b+t75Fjhb7AtLT tamil@ubuntu-delldesktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6tJ9y3solDCzFdXcczO8GJEwMqMYXWU5kuMz0rr/US6F72j0spSNmwGemMOZE9s7KtHgior/Ufw8/vMRkRhoBYB0C5KT9utGNqrtDdvH98h43XnKZ3K/yaHv/T+u2EYWNgHtOXMtgIVFnGwSD7RzF6TbwSfurprTV6ZUyyOxRcmHnkmwpbNwiwIoXB03k2PITdYCC3lEBEY0OC+vAW/E3+bFbV3RSEA0Cm31jmMyZyPeYiXiEnPXUIClsoDj34NwJTIgctwIsqM4eYUBg3pHliWYKEWDs55HZYoj3W/6Jw7MASzyAYFSmP8rQu8iS7tO1iEBRGOXFNxHirGQ1WqUn tmuthamizhan@MacBook-Air.local", "key_options": null, "keyfile": "/home/tamil/.ssh/authorized_keys", "manage_dir": true, "path": null, "s 2020-04-10T06:59:20.364 INFO:teuthology.task.ansible.out:tate": "present", "user": "tamil", "validate_certs": true} 2020-04-10T06:59:20.495 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n', u'name': u'jowilkin'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDYGVqKZgLx2LJ49x0kGo/Z5r8MVefSUCYhhrK/DK8ht+YK1AqLXErjAedgpVz0WuwirRjh5YkiXGMYiOdkMUYWGk5nwCHEKnnCpne7Xh1OOqge/QGHlUigWVH8VIHBsZ+wTW25mqOrgDSGB8+Qk7Dr2o1sLGUHcx7vITDWOrer4KjF2+H5TQH/7irUVORu6fzeGKtGyOrTwQ+ZT1+fuhfznDULFGyvKGdcXxxNOg+21Fssk2nCNTlgRCGjHSrfelfuMT7QEt+9puWd7MWXYQg7xx8jkJGMpEQRFxJTxnfUlUXf3x2lc9Q/SvKfpVMtc0N+bk2F9+euyvRvjtn4LJNx jowilkin@jowilkin.wilkins", "key_options": null, "keyfile": "/home/jowilkin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jowilkin", "validate_certs": true} 2020-04-10T06:59:20.517 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304', u'name': u'zyan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCtmxLK2rSomB14d9/fkAshmsXQfd6cJ+vEb0MZDaUT+OLmr1vAq0+H5g9w3hApJ2UEOFSaiz6ywgBAFeBeu/KmUALOhJIKKk0imoTunYclqxVPBK6tG/2b+ME8VE1afoEpMem3Bzfkrq0pDt6DLFBDlt9rRIqtO/ZlqEayWkEwTXJ7pwj+oJS03Z3Y7q0Iqw8PToKqJWwy2OvW1I9Pi0Q8s4YV9I34rSO7Wv4w43AXpFHRlvJWIcI5g5WFI5tnjel+kUAWMZft9ynFoAP5iXdByq897r4jAF74hapaxLMsHBz4MyYmPZqS76Cx2e3kGqXeetPinfcZxC3ViboR7o/1 root@zhyan-desk1", "key_options": null, "keyfile": "/home/zyan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zyan", "validate_certs": true} 2020-04-10T06:59:20.656 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4', u'name': u'bhubbard'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDP/yVttBEs7mpVfJL+reXb/BfB7DsLrA5ArVqEJjySi3NOz3e/wE8KRWbHD1UdraAU3P8b96GdAQxGOs4n0muOMejmaXWZCP3HkRL+rcd+03ZI1lb6ga9XjBARZfb2h+kg5jTJp/Ll6mwAYbSXvQ4MuDQzwGRDX5Q5Xme7OZ4Wb+vBmmkihHydySyP9YXbWWsBw4jvZCFSKMGsUVy/HIt1coZx/PyCG/K2Kl0nrE5ux2QqfdTIpzYo4c3y6uMIHDvt29lRmk3Gnh2//U5eN8mmRRKJlkvE8DKy3VYsl0JsP9pTWUZEYgV0EX251WTO0akQ2bKuszqd9Ai04RWnXUu5 brad@dhcp-1-165.bne.redhat.com", "key_options": null, "keyfile": "/home/bhubbard/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bhubbard", "validate_certs": true} 2020-04-10T06:59:20.678 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n', u'name': u'yuriw'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCw3MhMNDYjkiQADoC1mg9ga0Yj174FHIFt4RQIC8qK+6cB0N3CM5R4JwP8pR4tYkVFWmnYRScYK5VVdC873Ac2BzYxmse97zLJTbUnYWTdY+OFi1FUozsy3PwtiNHSd48Z/PBvkinQIey2QlMKaW3/yz9Of/Yfm4prxg9lBsd6/tOOFj3AsqIQ1OVhMfgeVEkdCL7AAbEYyMxePZ3UFrDwj05HDLgmFzx1xsNJFOuBwLfU/QNHoXJQrcVy0rZq9fB5rg74iz1JZs0uDWocQQK4Nd3Rv+wRGBuew2fmkrS2bQWXzMoSL+/i7pbf74LKcbWAE0wXEImqQeREHxq4ZZ5J yuriweinstein@Yuris-MacBook-Pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJUmq7btpBMjjODHXp1888xoBuX+rhiZg3ufuo69RgDxUMymJn3CPJ/5NGXoZ3SGAWKk0w0Dx6K/VO4y92mAV+cO81y8wZR4J3fPV3IbWRjN0X3IFoAvJi5n9v5hpG7z3RZCGmQnOZ99m1i9dge/3S8NpuPLIvYkSo9z00d+mNP6D8rGnqOmEBfEhUQXLeH7rbPwaMWPgB/hrpiPCyLR/+sGC5bq9PAsKnocqZ12cnvCtYxpnA7et6nL15ftEpUrKYAmRDxotuoUcsllP66GSszjbrgN6OBNDSqpGt/+RSJJwzZEGQ073gJuMveKy9tvD/PyEttjvVKL2KsBuXGSuL ubuntu@yw\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmXWgRRes8bzekTH2fTcxdAAquFn1SjPhw 2020-04-10T06:59:20.685 INFO:teuthology.task.ansible.out:s1Z9CkxNaqKe25XnzPehqrhg9etouUwvORQHdDJdTlmhTSYtn588Omq+1bvniab0KzAYgGZsWdPma/3E9qF7ChXIMik9Umip0drzYTaexCuC/Nu46brTxlnL0vaxnnVjDwmHC0CEJs4qN4e2wSlsA0uBDRWjxWFMdTUgSzII5xasHypUYZ0y3MNdbfY+SVF69DEDH8EOQMGYm8JQXNJFloQvmvLUFqT/63FsdpzRVTph4A8hvI1f8i6ULb7fPQN5ZS62wdXd/KvrBXrCvhOi7R63RM60mkMRXm1XajTGmCKiikfiqgTr yuriw@Yuris-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC41kRr6B/RC32DrQKRByalH832H6BpCQF9XdTvgYi4+LJt+kAuIGLDF4xm4uApTmWRrLspWm0Kksc0ger03uNx5qJqMEUnJvjxJnSDC0/XyxWapVxIviFkvv0BjbhVNOOaMU55ywrZ9hHgMdaFwuMuCOaB7f9Tm6IUXm2k82OXueeIG23Kziq+IQBysWOjvNUy/LLpl7l4+xOkhNUupUiQCqHvFH98VUdYWkHeyamafGVZmYbFLrOXgmviKo0u8LPxcT7psKLk0yFG7fG0DSak6DDdKJqb+G/NqMSPxRSAI4FbXQ6UOZb32Ozr1Sql9yfOXEYWZuatn7W5F8DjSv3p yuriw@yuriw-RH", "key_options": null, "keyfile": "/home/yuriw/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuriw", "validate_certs": true} 2020-04-10T06:59:20.815 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n', u'name': u'yehudasa'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAyRDKb+InDsnW4E/S12Pr6rK3wDfUG3j+iw/NsCWwN5xpJEA1/NhVlRzY8/jejc6ddmnNKWbm/UcYVrbo7yClRTGu45g0Qm9lmfvgDgi1k8mwUCy+edbQ/R8FB9cIHHCnUuWzRmE2XGaeDiSVQgiYqq3mOPY/hwe3G7jwFalYPsbmY1jNqpzODmKFMRd1bOwdHEOMTfh0owHWqhhXakU8em30lYraQVYH24FplH2NUBArxmTuctyfweHxx7yH3ZFcXFUpUMSLuVDWb1A7fMiL6eW6wiwh+dD+q5iFNvuioZITP2EjIsb9nS+eM9bOJZuZ+5pln2EsHee27LjunbwgWw== yehudasa@fatty\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDF3PxJYgI3tS7SG+RnUvxMxr0/XwBAio0oiP4AC+o8rUYYyPd/RU27TR1kphbuJhQWVtaZ6kFSBQeNQUxv6rb8hmMzvlP1FDEoGkjaOHOuYEc2DFkkEAfaR2AlNKn3pUQgLE00TLHctCgd57Cn2Ppt8xEi/vTuo0gX3xjWOL4N9QDElCXKNdxdyyJ/RNb/Mid+YxgKKjf8ppmhYsQ+Qqu4YwPzLkfK1Qb3YtR6Hbjb4jxC7C1Z13h0vUgHe7qSmUCYNo9eC+lEgNmmnLb2sXJaUUJM8MEex6v+4muWM9hV0cq/Wplrx8fhMWMa978sxH8yVkE3Svqc/K4ZWN59l3xT yehudasa@yehudasa-XPS-L401X\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDvplQk8xkSXF03uSJh3H4zyZHRMjvgUJBgGN27Kb7gAlIQv306wPKIprOwp+jDhEcPnEWBMRJMUJuBi0vEiqmDHt8mg8ZYcs+Gi7MbUgBokILdJuSG89KAgq7gBCuIhYbSg35pfvXvIZswiXvySqMrJNB495GYj/FrcWEBfQMuedMSkIfNzqPV9yR5YRh8fnSs3fjCzV96vzu7O4AzgwICbM6XmqVLXPax/32Hq0dz7sAawlQFCJYPHzVzjwm 2020-04-10T06:59:20.817 INFO:teuthology.task.ansible.out:ih46TRJnXrxewsRhk9GwKfI4SWaFIRdOdadGFSLgdZ+2iGE7hTZ3sRgkLOj+4zKa+xTbtjjXfw5up7J3hAmrK15Tz yehuda@yehuda-940X3G\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfNxvAw6cb33jg36vw8kJu48/BttmFJ4BFeanYqwRgtWZrKpAAGHEgCiNeyIttMrI3t7Spcdyp6dy6n0P3WUFVgHWEYtGqXwXqDC9UXISkUJVVDZoEynpGC92RghhzH53+7jX1Ytpr30r/xb1ktryTPCoEJRqyrg2j1b167i6PV5tuJCA+77gW5LWBmMXJCprjnfnJ92Gp8ZewBB6gL1x+3RFkMxGsibL0Qw1CTKfyAHNYc/jos6YiL0b42Ha+NmKwvzZ1GgIleRMleisSOyKPUlPdR7UnmtEXa9bl3Gc/hc7Zt+v4iUypr2WL0X4Yl7rJNH/AAd/Z6jPkAX9nHYzh yehudasa@yehudasa-desktop", "key_options": null, "keyfile": "/home/yehudasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yehudasa", "validate_certs": true} 2020-04-10T06:59:20.835 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n', u'name': u'tamil'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxrS30JmTZbnBd73s9PGZYxCkWH3u4En9Q0bYJan45W+j+y5vMzaNUVsp8We1tphqPQ9B0ygi80OxbHWu5gyv4g+M/JFtrnxv3QgBCYWeBWfh652Vbdlk1thl6btFnHUhVMSn5tkdnFxyVDJHIROoVXkHRaCNAUUy7tB60IhPAMomNcCGu9R8+opONFLXKTIfkYQ1cGbysr90DMs8ASEkvxdv40hIWhK4ufvL7CFFtgXuoC0c1WedBFOkr5LDegzL13xN97K9o5J8CAU0MUKk6H8KNKbQuNq1zANwymMFY599GIlxwai8iluJgfOObDRdCDmKM2f9RAEUXQM7oYoH7 tamil@tamil-VirtualBox\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvwiWVhNLeOLg72cpVIwbUYnmFh+Cqf/mE7Rc/Xci91on3GPrq++dC8t2GhTB5Zyq46EE27SmUkiZpHRnz7xwiJKM/tTgNEkffXDK3j0pA2E22iqV9FpfSA9Zdi1Puq7CPT7/rix4rF3SPvMtg/R1fkyornLFdzhJiMExCKJXGwf+QsKk4NVnr+nO3EEkKOoP1wio1bceZ53rCt9Tyu8NawAB2nL2fvtBPQDAriQ8bLX69XLbOfhFI6GJo815/j6CQxNmvA0l5ABJc8+A71jZmndzumFRgqBuhiCM5ohsGR0JGahUQjFEA4quNTx8EWTGRsqhAYszzpqJzuN+Xb6tJ tamil@macbookair\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEPzEyDGI7poyXVPG70vgQpvlU/lTt6Vvv5CoHWJoSrHklDdJEqekhCB87BX4DTrqbb5xoM7yfxle/z9UzgVwSUWqsPdZHA4EbrNFRKFxi/gc2YrygVSjumTyzf9WJJ5vCZvTQexOj7rjWVnJkjXHKMXZXIAQN/bJf6J21t+4A3XLrl30/4XXyq664NONUNjBHxHtlbS2LM6Th/hK4B23oj2BugX2j/50vWgd9H36bPr8+JGskOi6MH2ndZv5pYtLznFzQmsbixjaN8kCuyQdlRCOndPEFoHFhDla2MeB+h3XQLDXW7J1TR2URFSfcNq5ndDS4i0b+t75Fjhb7AtLT tamil@ubuntu-delldesktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6tJ9y3solDCzFdXcczO8GJEwMqMYXWU5kuMz0rr/US6F72j0spSNmwGemMOZE9s7KtHgior/Ufw8/vMRkR 2020-04-10T06:59:20.837 INFO:teuthology.task.ansible.out:hoBYB0C5KT9utGNqrtDdvH98h43XnKZ3K/yaHv/T+u2EYWNgHtOXMtgIVFnGwSD7RzF6TbwSfurprTV6ZUyyOxRcmHnkmwpbNwiwIoXB03k2PITdYCC3lEBEY0OC+vAW/E3+bFbV3RSEA0Cm31jmMyZyPeYiXiEnPXUIClsoDj34NwJTIgctwIsqM4eYUBg3pHliWYKEWDs55HZYoj3W/6Jw7MASzyAYFSmP8rQu8iS7tO1iEBRGOXFNxHirGQ1WqUn tmuthamizhan@MacBook-Air.local", "key_options": null, "keyfile": "/home/tamil/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tamil", "validate_certs": true} 2020-04-10T06:59:20.950 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4', u'name': u'dang'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDx37xbP0jYqopygG66LbSMRPoqpGUPzQZ3G8eqJCk2I dang@revan.aa.redhat.com-2015-07-17", "key_options": null, "keyfile": "/home/dang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dang", "validate_certs": true} 2020-04-10T06:59:20.982 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n', u'name': u'jowilkin'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDYGVqKZgLx2LJ49x0kGo/Z5r8MVefSUCYhhrK/DK8ht+YK1AqLXErjAedgpVz0WuwirRjh5YkiXGMYiOdkMUYWGk5nwCHEKnnCpne7Xh1OOqge/QGHlUigWVH8VIHBsZ+wTW25mqOrgDSGB8+Qk7Dr2o1sLGUHcx7vITDWOrer4KjF2+H5TQH/7irUVORu6fzeGKtGyOrTwQ+ZT1+fuhfznDULFGyvKGdcXxxNOg+21Fssk2nCNTlgRCGjHSrfelfuMT7QEt+9puWd7MWXYQg7xx8jkJGMpEQRFxJTxnfUlUXf3x2lc9Q/SvKfpVMtc0N+bk2F9+euyvRvjtn4LJNx jowilkin@jowilkin.wilkins", "key_options": null, "keyfile": "/home/jowilkin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jowilkin", "validate_certs": true} 2020-04-10T06:59:21.088 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb', u'name': u'branto'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2YyDoeKdaCuJif79AxJAorqpUj33WHbHu/eIWpHju+iIF3f94K0qyEXgs4AwZin9pzfyd8N1aKNT8ZFPss6L0Xk68aO/6tdInq+1lTMMUC5v/LBqsIV2btPTFHrQWA0R10v05aZFvFaSqhHNsZaj98vQV7P+jiSs3yPWqCIWUTQ1VXnCeQnB4R5tob0gpMuonhkKq7vrJPhf6ge9Av+yjL7+8oP5LJqFJ5npDxkwZ0u3L0QQ9ftAT/e7Y7ESpwpgaI3Trl0EwAXFmiLjXEUA+FBbE7ko9kLqXqRjc1C0BdRG9sO0QKBGDBEx1W3w8rCPKgaDlVyqsBT1yvIjqqXDOQ== borix@dhcp-31-190.brq.redhat.com", "key_options": null, "keyfile": "/home/branto/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "branto", "validate_certs": true} 2020-04-10T06:59:21.152 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4', u'name': u'bhubbard'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDP/yVttBEs7mpVfJL+reXb/BfB7DsLrA5ArVqEJjySi3NOz3e/wE8KRWbHD1UdraAU3P8b96GdAQxGOs4n0muOMejmaXWZCP3HkRL+rcd+03ZI1lb6ga9XjBARZfb2h+kg5jTJp/Ll6mwAYbSXvQ4MuDQzwGRDX5Q5Xme7OZ4Wb+vBmmkihHydySyP9YXbWWsBw4jvZCFSKMGsUVy/HIt1coZx/PyCG/K2Kl0nrE5ux2QqfdTIpzYo4c3y6uMIHDvt29lRmk3Gnh2//U5eN8mmRRKJlkvE8DKy3VYsl0JsP9pTWUZEYgV0EX251WTO0akQ2bKuszqd9Ai04RWnXUu5 brad@dhcp-1-165.bne.redhat.com", "key_options": null, "keyfile": "/home/bhubbard/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bhubbard", "validate_certs": true} 2020-04-10T06:59:21.215 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6', u'name': u'xiaoxichen'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDO00mus2XPX/EfUZ/YsKI84TnWj5TbZP7Fs01EkoOTPgNoYysVdf0PwhAooz1qPrv+HUsUMpQT6uNYfCHyqx9rScR+XUlyQuFe7Sjp/dn3y+b1PKa6G4HX7f1eejR3ZaLZXA4R/8oucAXqQS5pDr3Y7prwkrp9cyk8uawBXoDcAlA1SMIPZPrtO1+iZw20NDhixsG7NE5jMFfw3q8xTckZEqBbteybci7X1v4lEYXCVlbSbUhyZ2Af8HEj09JBDOL2bI3yLvj9a5Oj/HtKubyApzZpA7rgeoii2UZQGuSRI1VJktp9YC0zGk5S843xYW92utbALKyQ6DdB2F5gc0DX xiaoxchen@LM-SHC-16501223", "key_options": null, "keyfile": "/home/xiaoxichen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiaoxichen", "validate_certs": true} 2020-04-10T06:59:21.278 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n', u'name': u'yehudasa'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAyRDKb+InDsnW4E/S12Pr6rK3wDfUG3j+iw/NsCWwN5xpJEA1/NhVlRzY8/jejc6ddmnNKWbm/UcYVrbo7yClRTGu45g0Qm9lmfvgDgi1k8mwUCy+edbQ/R8FB9cIHHCnUuWzRmE2XGaeDiSVQgiYqq3mOPY/hwe3G7jwFalYPsbmY1jNqpzODmKFMRd1bOwdHEOMTfh0owHWqhhXakU8em30lYraQVYH24FplH2NUBArxmTuctyfweHxx7yH3ZFcXFUpUMSLuVDWb1A7fMiL6eW6wiwh+dD+q5iFNvuioZITP2EjIsb9nS+eM9bOJZuZ+5pln2EsHee27LjunbwgWw== yehudasa@fatty\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDF3PxJYgI3tS7SG+RnUvxMxr0/XwBAio0oiP4AC+o8rUYYyPd/RU27TR1kphbuJhQWVtaZ6kFSBQeNQUxv6rb8hmMzvlP1FDEoGkjaOHOuYEc2DFkkEAfaR2AlNKn3pUQgLE00TLHctCgd57Cn2Ppt8xEi/vTuo0gX3xjWOL4N9QDElCXKNdxdyyJ/RNb/Mid+YxgKKjf8ppmhYsQ+Qqu4YwPzLkfK1Qb3YtR6Hbjb4jxC7C1Z13h0vUgHe7qSmUCYNo9eC+lEgNmmnLb2sXJaUUJM8MEex6v+4muWM9hV0cq/Wplrx8fhMWMa978sxH8yVkE3Svqc/K4ZWN59l3xT yehudasa@yehudasa-XPS-L401X\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDvplQk8xkSXF03uSJh3H4zyZHRMjvgUJBgGN27Kb7gAlIQv306wPKIprOwp+jDhEcPnEWBMRJMUJuBi0vEiqmDHt8mg8ZYcs+Gi7MbUgBokILdJuSG89KAgq7gBCuIhYbSg35pfvXvIZswiXvySqMrJNB495GYj/FrcWEBfQMuedMSkIfNzqPV9yR5YRh8fnSs3fjCzV96vzu7O4AzgwICbM6XmqVLXPax/32Hq0dz7sAawlQFCJYPHzVzjwm 2020-04-10T06:59:21.280 INFO:teuthology.task.ansible.out:ih46TRJnXrxewsRhk9GwKfI4SWaFIRdOdadGFSLgdZ+2iGE7hTZ3sRgkLOj+4zKa+xTbtjjXfw5up7J3hAmrK15Tz yehuda@yehuda-940X3G\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfNxvAw6cb33jg36vw8kJu48/BttmFJ4BFeanYqwRgtWZrKpAAGHEgCiNeyIttMrI3t7Spcdyp6dy6n0P3WUFVgHWEYtGqXwXqDC9UXISkUJVVDZoEynpGC92RghhzH53+7jX1Ytpr30r/xb1ktryTPCoEJRqyrg2j1b167i6PV5tuJCA+77gW5LWBmMXJCprjnfnJ92Gp8ZewBB6gL1x+3RFkMxGsibL0Qw1CTKfyAHNYc/jos6YiL0b42Ha+NmKwvzZ1GgIleRMleisSOyKPUlPdR7UnmtEXa9bl3Gc/hc7Zt+v4iUypr2WL0X4Yl7rJNH/AAd/Z6jPkAX9nHYzh yehudasa@yehudasa-desktop", "key_options": null, "keyfile": "/home/yehudasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yehudasa", "validate_certs": true} 2020-04-10T06:59:21.352 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e', u'name': u'ffilz'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz7e2G3Ziaus2yMviKh9s65e+yaqy1+ezs5IC6US90/8WumNAWqC4Yr3ckvCc120r3VwpJWcvaAWCF+4IZeO43lSNLadbCnJONed5YklcUuIJvjC1eDbZTMLJ7h0ijgNXgAFmFq0AV2NHuE2L2VKXk2hsIqDVGjZLUpi7W+ic07xnScEoXXDSk5mhJ6bKmFsUYCC91g2roYs9Ui5fGd+ey28SFOADI7osDLpLkiTF2OzGel93o5qobNZwKR0AxGsBIta17mvf3I3307bGnvugDXpNB0FaS1QZhIzMO4G4LZL9zahCvv4wm5GmuO4F0+Ac7dwFu3KZKtHQe3g6WHuAv ffilz@redhat.com", "key_options": null, "keyfile": "/home/ffilz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ffilz", "validate_certs": true} 2020-04-10T06:59:21.440 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4', u'name': u'dang'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDx37xbP0jYqopygG66LbSMRPoqpGUPzQZ3G8eqJCk2I dang@revan.aa.redhat.com-2015-07-17", "key_options": null, "keyfile": "/home/dang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dang", "validate_certs": true} 2020-04-10T06:59:21.482 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n', u'name': u'joao'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpGZK2cWUtE85Bt7fl+kSeiC0lM/EN+H2X3ePGq3LiD4g5R5n6nJsLaJaIZXdJ96G0JSnj07G8Xu1CkVbGDHuVg2O9tnxLhz6tzDwFYNFrQIpALlQwOq8OiwGDDyFkMXQ3JqCCWvyD0HLysOJQIBIlb/o92ZCSAGOyNX1JjrhXF+mlBk1AUW11UeZOeGZQ34MKHapH7T/DuexCz5gfZI/jY9Rg3hr4jb1zGttvAF2eeLJCjqD/Vi4mowYB/Kn/PJhNixeA2sli2i8r6yO/bFmA/PWnfSl+EqD4mUJWb5ATeJ3hyW8qVLOogY6XVu4Sd1VgL7l1RYzXuxJV3Rb1Yw2T joao@sepia\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDV0jbKpbmXW1bhRIOBIdVV1L73qjls7s5pxiW2EMPvZLYzxLBE92e3Vf8bSMWlm1Lyb7a+u5BR85+2c+h2F59sgU1Saek5ebPpFU5qBn8AzYH18iMVSPciByOzwS0NXwWnksckWxUkQgP9bZP/1ORQ7jWZEOHZ5mEQ0oMkP0bfG1cpFrNTxOhDmwbCiQ8dp6KeZaMg4hErALr5Eh/EESxKIkuwQzk9bDbtjPRDu0F2dE0NCjs0k0BXmDA80GksxuOacRxE95K6/er11qst/VqA7o6BfhxJMTXWMdj8NhzIUNjZ8LqLTG6P/XGR24sA2d2WBhlEKRMZNqeEjzmi1REr joao@tagide", "key_options": null, "keyfile": "/home/joao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joao", "validate_certs": true} 2020-04-10T06:59:21.589 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb', u'name': u'branto'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2YyDoeKdaCuJif79AxJAorqpUj33WHbHu/eIWpHju+iIF3f94K0qyEXgs4AwZin9pzfyd8N1aKNT8ZFPss6L0Xk68aO/6tdInq+1lTMMUC5v/LBqsIV2btPTFHrQWA0R10v05aZFvFaSqhHNsZaj98vQV7P+jiSs3yPWqCIWUTQ1VXnCeQnB4R5tob0gpMuonhkKq7vrJPhf6ge9Av+yjL7+8oP5LJqFJ5npDxkwZ0u3L0QQ9ftAT/e7Y7ESpwpgaI3Trl0EwAXFmiLjXEUA+FBbE7ko9kLqXqRjc1C0BdRG9sO0QKBGDBEx1W3w8rCPKgaDlVyqsBT1yvIjqqXDOQ== borix@dhcp-31-190.brq.redhat.com", "key_options": null, "keyfile": "/home/branto/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "branto", "validate_certs": true} 2020-04-10T06:59:21.653 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n', u'name': u'nhm'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHhYb/mGgHbEntAN2iPFTV1k8MYDs+jtsJDPE9MWtUZXksdeG8cEfXDBGhiBsprjELsW4+czJBqqESugFMevEysTVKIo5lc95bf1nApxfiRyMnAxb5wQ2OeLY62x26TWJhPu6HMajV+gb8CKjxDobf1Ix4ELbYlL77SOH1VVDtVZ6HiwtYngXAPMONCdI99cuwJlBpv5RjDoLwljOoxEXIh+1Fsvth+iymp0rIPuJx91DzYYgWZFfCLPHRgDy0H7YRdQWUzhrxTyzuT27vsNh5L8HgdowzOu9f2HwGzlTlxli94IJLMBY2359xsqjSswZzAv+BiPDAZMUwJpYvRvSt nhm@clusterfaq.org\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1JAKymiRDyVar5P75cMycxz9u8VsLuNtf/0t1na4ofa4YmanKswPQQz1Am82pvyPqXlBbdIGVneyMrkEoFAk4nTtfmCl7WdtaxnH87Dr9BOLEp+lIPHy7jpS/dcVmKRPB1qCYZ5GNAZ9g+DrXdaeOJdUOpIK49BP1IK8Uw5eBWAx/ae6/12yz+d/6e9kBvO7OUamMSne6Ghe2pRoI8jFOdWMbQy3ofb5wSHZGPNDn0YTGbNWHlfEAcrFmGQSTwohca4SqCJ1NgKnYu4qPVmb3Gbpm+MpUPs8Gy++rQ8wVZ5xWZW/yReIbJCqqVY4vrQzVywQXRucIrB1VyvShuNW3 nhm@metropolis\nssh-dss AAAAB3NzaC1kc3MAAACBAM7BfQAinQmLlkTgt7IIg3nQyyaJRiWA4agfu/HncrcAmepGqPCwy451g1eLMD3AC4QptHnV80Y47YQTyX1VQixIQ6M9zHG6Bo7IXzyqFHFpqnDZN25DaMM4na5bf1CwC9RtR7VWubojatPylOpflkChUBuAGWnzwkz2cxl4Dc47AAAAFQD35l/Lg4eTqZ83eTgJ5lihzHS/awAAAIB/ZdMSE6M0zEvdNPsWY28bvQ0QXN7w1ncVXGachA1JOEFGYvsbHeoywLpSYoiPLV57/wZT2dUzG70u80IQGildZ1WgQNn7C9r2Z3ZUgKmRGyC+PpYEa6nv+U4CHdMViBRXxNrP0bj2 2020-04-10T06:59:21.654 INFO:teuthology.task.ansible.out:CfQvUxUxrHWtOu/da2W23cpSs6Wz8gtz2QAAAIEAvf4vx/32Sy+kbvqOCYz0wu0ItlECN0qOk3gAMvphYMI2sW2ez2+Tsua4eoPRXT10zFxiRTkHQd5FSIvNWXV2iqed6rpwTu5K8vJz2duMIyI+xWLHo4XudtZdNxggkf+RW9JvPOMLMp4sCzBXxpreHBsb1CynndKeYVrgMLybZ28= mark@Marks-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfDOM2v4h6Dk4pMWPQxxHI5rhFtLjkme/4hAp6ki/kJ1XOSzBQdQmD2UPDKvfO9lKO7v12G2Wt+re+0giAcSkfej4+xqV53u7d2pTbMI5U3zohZt8AgESx22DAxj1wBZYuyGO2TRI7DRQMxC47E8EJgcmZBBnMKk1Rz8k71pTmg+CI2vjWx/2x2bup9Mqe4uMA8ubjg2qRlWEqPKIKcHZyrAPdIlpy2FSZ9SPa0mJpdl6aYcFNgonD8BO6A0u2RPO90KmEWrRWu69vRvbbTDiqE0CkJJlAlu3jeFVD8US9vymBHSI0BDFUbukuyG1tAujytqGBRLzxQDU9JOYMP45v nhm@mocha", "key_options": null, "keyfile": "/home/nhm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nhm", "validate_certs": true} 2020-04-10T06:59:21.750 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6', u'name': u'xiaoxichen'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDO00mus2XPX/EfUZ/YsKI84TnWj5TbZP7Fs01EkoOTPgNoYysVdf0PwhAooz1qPrv+HUsUMpQT6uNYfCHyqx9rScR+XUlyQuFe7Sjp/dn3y+b1PKa6G4HX7f1eejR3ZaLZXA4R/8oucAXqQS5pDr3Y7prwkrp9cyk8uawBXoDcAlA1SMIPZPrtO1+iZw20NDhixsG7NE5jMFfw3q8xTckZEqBbteybci7X1v4lEYXCVlbSbUhyZ2Af8HEj09JBDOL2bI3yLvj9a5Oj/HtKubyApzZpA7rgeoii2UZQGuSRI1VJktp9YC0zGk5S843xYW92utbALKyQ6DdB2F5gc0DX xiaoxchen@LM-SHC-16501223", "key_options": null, "keyfile": "/home/xiaoxichen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiaoxichen", "validate_certs": true} 2020-04-10T06:59:21.783 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313', u'name': u'nwatkins'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaetIvR7YgpCq142fBQtOYsJrOMW+Wxu1beNXPJ8qzT5MxD05sdKntVE0DPveUydaeI8/vHeNjXDKN8mt1eO7Lf/rDKHOW/Y/36bUfQAODru9sdNCEJfSPgq1y6+X1ISljvLBXOOl+nQXlx3n5OjThNQbmdthgCT2Fk4hEoK7gn4+ol6foTkTdUXIBam8LuPU6f1KIG6ae1oAVFWLeavAbvmxUTmVl04+hKJWSPnvZxWupaPuiHvsImaQ7GwBjKhzL1wSMly0+pa2zbVYZKkLBA7Fx6fzP7jcEieOOPM1YU0jR5JF+WVzeRFJ1V9n75yOw6S4ef/X/zwWx42cdfx3N nwatkins@martini\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDRivdSX6cbPrdMrjz1oYgn0hb8VQo+QU7wD+AcCQi/CH06ny+2QjSr7bfdpgQ81Wgwqt9+e3KcLD5qdt3tneC8sgSAR5QUQrWKhPejrqdV+3990k8k0KoKIOZ9MGFc1FBv5IpuT/2d535iHJb0ZEg8mFKqKDw71sBEoQT4WElatf5DeVC507hP7GnrhZgtTPcrVddOLkmmqTQtLE1zvo9Y/eel8O2JMImMd6mdVBIcFQ1alU1wV4jLdMpagoPelvVTVeNkGRnOgvvQac5RXtGSMP1R5PoGz+1FgRhDCtdTXRfOdtrlJPtUjYiGPpIoKIUymlvQGjUrrK3WiQIIkIfDtFFu48EN4drjgQpCEQ2D0NMKEmk74mjx3CJDOF1Nj/2Fa1kjECiz4EM7sR8dDG5cW5Y8XsDoomgugM+rsOH9kWFFClHUJmdlNGWLhJnu9UQHLYZjQw3zT0F+FXUsUxKJUVTUdxQx5HBO9QFsSe13NPDnX64+SQ2VVStt8oQAvFO+6260zbux0RVaJM5dTNwnIHD/uI3PeR1z2b+iKhyj/9VP2uBRpGydLw/etQu/lS2K30S+kZIDAp0sqwXH89bdq3AJBipcryrnW1Ov9K5Mfl1UlV95/MivrnOFISWN1F9XR1kuIIYdCkI7hq/FHZ9gd84AjTiERfxRv8VCIHBVEQ== nwatkins@redhat.com", "key_options": null, "keyfile": "/home/nwatkins/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nwatkins", "validate_certs": true} 2020-04-10T06:59:21.895 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e', u'name': u'ffilz'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz7e2G3Ziaus2yMviKh9s65e+yaqy1+ezs5IC6US90/8WumNAWqC4Yr3ckvCc120r3VwpJWcvaAWCF+4IZeO43lSNLadbCnJONed5YklcUuIJvjC1eDbZTMLJ7h0ijgNXgAFmFq0AV2NHuE2L2VKXk2hsIqDVGjZLUpi7W+ic07xnScEoXXDSk5mhJ6bKmFsUYCC91g2roYs9Ui5fGd+ey28SFOADI7osDLpLkiTF2OzGel93o5qobNZwKR0AxGsBIta17mvf3I3307bGnvugDXpNB0FaS1QZhIzMO4G4LZL9zahCvv4wm5GmuO4F0+Ac7dwFu3KZKtHQe3g6WHuAv ffilz@redhat.com", "key_options": null, "keyfile": "/home/ffilz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ffilz", "validate_certs": true} 2020-04-10T06:59:21.926 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22', u'name': u'mkidd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJD+2BHStkaMJ8RStIIz+yDJCGNT1wbwUnmuO71vNlFxzIyHGvCq3ktx2xTwZz/kj6ZayXYWIRH//BhK2/iK/v+u2WQMBURfMHbf0lDMmNfbHCDBjYvEYgFo4Jwvzq4WHus0AHh9TdVt+TpcxyGPhVbUJh8NZut+w4cRIKu2xGoZZGDeLgv9ijXvDPrCdBxtpvQnF99C0bCTdPM7MK1FSCQEaRmNUgzEj0H7R+gmWi0T4HUD8ZY7y0vWbYPqT3Z07My1u2BfRBYpXnv1655/b777TkSlzSasORhSTT0AkW25u90Rsj2JWr39+49/Ia9uOyFtZIirugEv2v7v/l0grN linuxkidd@zenbook", "key_options": null, "keyfile": "/home/mkidd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkidd", "validate_certs": true} 2020-04-10T06:59:22.033 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n', u'name': u'joao'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpGZK2cWUtE85Bt7fl+kSeiC0lM/EN+H2X3ePGq3LiD4g5R5n6nJsLaJaIZXdJ96G0JSnj07G8Xu1CkVbGDHuVg2O9tnxLhz6tzDwFYNFrQIpALlQwOq8OiwGDDyFkMXQ3JqCCWvyD0HLysOJQIBIlb/o92ZCSAGOyNX1JjrhXF+mlBk1AUW11UeZOeGZQ34MKHapH7T/DuexCz5gfZI/jY9Rg3hr4jb1zGttvAF2eeLJCjqD/Vi4mowYB/Kn/PJhNixeA2sli2i8r6yO/bFmA/PWnfSl+EqD4mUJWb5ATeJ3hyW8qVLOogY6XVu4Sd1VgL7l1RYzXuxJV3Rb1Yw2T joao@sepia\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDV0jbKpbmXW1bhRIOBIdVV1L73qjls7s5pxiW2EMPvZLYzxLBE92e3Vf8bSMWlm1Lyb7a+u5BR85+2c+h2F59sgU1Saek5ebPpFU5qBn8AzYH18iMVSPciByOzwS0NXwWnksckWxUkQgP9bZP/1ORQ7jWZEOHZ5mEQ0oMkP0bfG1cpFrNTxOhDmwbCiQ8dp6KeZaMg4hErALr5Eh/EESxKIkuwQzk9bDbtjPRDu0F2dE0NCjs0k0BXmDA80GksxuOacRxE95K6/er11qst/VqA7o6BfhxJMTXWMdj8NhzIUNjZ8LqLTG6P/XGR24sA2d2WBhlEKRMZNqeEjzmi1REr joao@tagide", "key_options": null, "keyfile": "/home/joao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joao", "validate_certs": true} 2020-04-10T06:59:22.076 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af', u'name': u'jlopez'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrRvNeBa+/GXxkSDmaim7U1wFpdBvgcK9/eP1I4fPS5xAWyxXI0s0ryX4Jwb7auhZdcqPjgUPQvIVVRridKJPz7l6jUlbIqdnZu4VHuaGG/DfIVxsSmw5aI5qI9Fg/s16JPwOUuzcLHsSSh8tFxY9L1yvs3ztQnKHW6nN5+zhPbW8UjmTNUksXXAeCM/OZn+IOOzzvfh0vVe/mghIW2JpOjHkZv4qwaWYcHg6+wWE4m8l84VIegDBiONknZXcQaRj82nHtfdnb2FSiV4+gKP18i90ai0iTKKXUkMJNFKp1T5MuF7UuJLA7l10FmgQJsZBS2Loa5oeqbHYeztNAfUxF JCLO@oh-mbp-jcl", "key_options": null, "keyfile": "/home/jlopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlopez", "validate_certs": true} 2020-04-10T06:59:22.192 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n', u'name': u'nhm'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHhYb/mGgHbEntAN2iPFTV1k8MYDs+jtsJDPE9MWtUZXksdeG8cEfXDBGhiBsprjELsW4+czJBqqESugFMevEysTVKIo5lc95bf1nApxfiRyMnAxb5wQ2OeLY62x26TWJhPu6HMajV+gb8CKjxDobf1Ix4ELbYlL77SOH1VVDtVZ6HiwtYngXAPMONCdI99cuwJlBpv5RjDoLwljOoxEXIh+1Fsvth+iymp0rIPuJx91DzYYgWZFfCLPHRgDy0H7YRdQWUzhrxTyzuT27vsNh5L8HgdowzOu9f2HwGzlTlxli94IJLMBY2359xsqjSswZzAv+BiPDAZMUwJpYvRvSt nhm@clusterfaq.org\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1JAKymiRDyVar5P75cMycxz9u8VsLuNtf/0t1na4ofa4YmanKswPQQz1Am82pvyPqXlBbdIGVneyMrkEoFAk4nTtfmCl7WdtaxnH87Dr9BOLEp+lIPHy7jpS/dcVmKRPB1qCYZ5GNAZ9g+DrXdaeOJdUOpIK49BP1IK8Uw5eBWAx/ae6/12yz+d/6e9kBvO7OUamMSne6Ghe2pRoI8jFOdWMbQy3ofb5wSHZGPNDn0YTGbNWHlfEAcrFmGQSTwohca4SqCJ1NgKnYu4qPVmb3Gbpm+MpUPs8Gy++rQ8wVZ5xWZW/yReIbJCqqVY4vrQzVywQXRucIrB1VyvShuNW3 nhm@metropolis\nssh-dss AAAAB3NzaC1kc3MAAACBAM7BfQAinQmLlkTgt7IIg3nQyyaJRiWA4agfu/HncrcAmepGqPCwy451g1eLMD3AC4QptHnV80Y47YQTyX1VQixIQ6M9zHG6Bo7IXzyqFHFpqnDZN25DaMM4na5bf1CwC9RtR7VWubojatPylOpflkChUBuAGWnzwkz2cxl4Dc47AAAAFQD35l/Lg4eTqZ83eTgJ5lihzHS/awAAAIB/ZdMSE6M0zEvdNPsWY28bvQ0QXN7w1ncVXGachA1JOEFGYvsbHeoywLpSYoiPLV57/wZT2dUzG70u80IQGildZ1WgQNn7C9r2Z3ZUgKmRGyC+PpYEa6nv+U4CHdMViBRXxNrP0bj2 2020-04-10T06:59:22.194 INFO:teuthology.task.ansible.out:CfQvUxUxrHWtOu/da2W23cpSs6Wz8gtz2QAAAIEAvf4vx/32Sy+kbvqOCYz0wu0ItlECN0qOk3gAMvphYMI2sW2ez2+Tsua4eoPRXT10zFxiRTkHQd5FSIvNWXV2iqed6rpwTu5K8vJz2duMIyI+xWLHo4XudtZdNxggkf+RW9JvPOMLMp4sCzBXxpreHBsb1CynndKeYVrgMLybZ28= mark@Marks-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfDOM2v4h6Dk4pMWPQxxHI5rhFtLjkme/4hAp6ki/kJ1XOSzBQdQmD2UPDKvfO9lKO7v12G2Wt+re+0giAcSkfej4+xqV53u7d2pTbMI5U3zohZt8AgESx22DAxj1wBZYuyGO2TRI7DRQMxC47E8EJgcmZBBnMKk1Rz8k71pTmg+CI2vjWx/2x2bup9Mqe4uMA8ubjg2qRlWEqPKIKcHZyrAPdIlpy2FSZ9SPa0mJpdl6aYcFNgonD8BO6A0u2RPO90KmEWrRWu69vRvbbTDiqE0CkJJlAlu3jeFVD8US9vymBHSI0BDFUbukuyG1tAujytqGBRLzxQDU9JOYMP45v nhm@mocha", "key_options": null, "keyfile": "/home/nhm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nhm", "validate_certs": true} 2020-04-10T06:59:22.214 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb', u'name': u'haomaiwang'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDFkUz8Wr2RPiY8+xf2hL4qMT+jA4mo1gmpIF8q7uVR03vX5NhzQVxVnXiyLCFv6susSMxpBafa7aLutqigXhDGJWUbsJII5yTnFYVer0QmudEsLmoDyvI1HCNk1IVY6aIESE50N3Ndx+agWasIU8TGKqJdDMPyiUtfrvsrGWi/91ZLX6aJSrFqBJtTpxDoNe4XL6QRPNv0wb8GF0az5graXkzVe3GkaQS9lyw8m2ougDvW1SLvoeE+lhUUGhJGkkCZKv1f+QNLhUHyQ3XkXqh4MrP3pPUKIuQ7nKiWDjCP6Db+ZcejjTRC19D2gLjhrw3Z2J7nn/loijlFzV5dp+BL haomaiwang@gmail.com", "key_options": null, "keyfile": "/home/haomaiwang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "haomaiwang", "validate_certs": true} 2020-04-10T06:59:22.365 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608', u'name': u'jdillaman'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA296in+8Sr9slF8hrzAV3dNz0q94Q4FlFaMYw08II5PQm6HOpabGOGOwOud/PPeh3lMUHhs45N5WnbgdfQdzz9lpDoNBcqxeACdMicdECMHtYWf2GuFUdTAnPBFVl8wFW5JcziQYJePUcAUJRXGGfdMLMppAi8VEuhMw/z3jz89UN+zf51Y9eT3rgAVSCR+Zn47kite4OB2S1aVMEcZBE0fPnh60i+rpN26s80fh3wG2N9JrPBEvst6LbL0aEixrlRUtoFHZuWNCIjntVMANa+idpuWSxN2dzfcoxpGLF1TFzqx0moznp0PnDs7ZAFFS1/EkeSCgBflyhPV6csG3VNw== jdillaman@ceph1.example.com", "key_options": null, "keyfile": "/home/jdillaman/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jdillaman", "validate_certs": true} ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313', u'name': u'nwatkins'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaetIvR7YgpCq142fBQtOYsJrOMW+Wxu1beNXPJ8qzT5MxD05sdKntVE0DPveUydaeI8/vHeNjXDKN8mt1eO7Lf/rDKHOW/Y/36bUfQAODru9sdNCEJfSPgq1y6+X1ISljvLBXOOl+nQXlx3n5OjThNQbmdthgCT2Fk4hEoK7gn4+ol6foTkTdUXIBam8LuPU6f1KIG6ae1oAVFWLeavAbvmxUTmVl04+hKJWSPnvZxWupaPuiHvsImaQ7GwBjKhzL1wSMly0+pa2zbVYZKkLBA7Fx6fzP7jcEieOOPM1YU0jR5JF+WVzeRFJ1V9n75yOw6S4ef/X/zwWx42cdfx3N nwatkins@martini\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDRivdSX6cbPrdMrjz1oYgn0hb8VQo+QU7wD+AcCQi/CH06ny+2QjSr7bfdpgQ81Wgwqt9+e3KcLD5qdt3tneC8sgSAR5QUQr 2020-04-10T06:59:22.367 INFO:teuthology.task.ansible.out:WKhPejrqdV+3990k8k0KoKIOZ9MGFc1FBv5IpuT/2d535iHJb0ZEg8mFKqKDw71sBEoQT4WElatf5DeVC507hP7GnrhZgtTPcrVddOLkmmqTQtLE1zvo9Y/eel8O2JMImMd6mdVBIcFQ1alU1wV4jLdMpagoPelvVTVeNkGRnOgvvQac5RXtGSMP1R5PoGz+1FgRhDCtdTXRfOdtrlJPtUjYiGPpIoKIUymlvQGjUrrK3WiQIIkIfDtFFu48EN4drjgQpCEQ2D0NMKEmk74mjx3CJDOF1Nj/2Fa1kjECiz4EM7sR8dDG5cW5Y8XsDoomgugM+rsOH9kWFFClHUJmdlNGWLhJnu9UQHLYZjQw3zT0F+FXUsUxKJUVTUdxQx5HBO9QFsSe13NPDnX64+SQ2VVStt8oQAvFO+6260zbux0RVaJM5dTNwnIHD/uI3PeR1z2b+iKhyj/9VP2uBRpGydLw/etQu/lS2K30S+kZIDAp0sqwXH89bdq3AJBipcryrnW1Ov9K5Mfl1UlV95/MivrnOFISWN1F9XR1kuIIYdCkI7hq/FHZ9gd84AjTiERfxRv8VCIHBVEQ== nwatkins@redhat.com", "key_options": null, "keyfile": "/home/nwatkins/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nwatkins", "validate_certs": true} 2020-04-10T06:59:22.486 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3', u'name': u'kchai'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAuTvkaUtEaLbLcYA9Zw3MoHI5ZPudKf6CAbAZdhp4RMVujimR61TT6LdRF20g6F7pH7GrI/1WP2bu7a3sZyCZ1nCJCOhrTTS+2wwy7oGo/0yY4l+Bw6i7LGmJgqpO5dDxwADMlaHUwJ7Rf1Q+Pd7NmggkAVTmaN6qUGffz9hZsj7ibD/Z8/GFtQsCEMloTLtnz4NB6Gxe54UXEKz9pvjpXBhs0xWsJtDKaPFrbKRKswuH45XPuwJhGr249ZYtwQqcgXSWhOUcPTwf5Y6fj27aL2dx566DxQzqzNbcTcO96MguIy8Ufnc0kIOQQw9oHJZtg82zwM6+2mWr9J/K94Gv9w== kefu@save.node2", "key_options": null, "keyfile": "/home/kchai/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchai", "validate_certs": true} 2020-04-10T06:59:22.508 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22', u'name': u'mkidd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJD+2BHStkaMJ8RStIIz+yDJCGNT1wbwUnmuO71vNlFxzIyHGvCq3ktx2xTwZz/kj6ZayXYWIRH//BhK2/iK/v+u2WQMBURfMHbf0lDMmNfbHCDBjYvEYgFo4Jwvzq4WHus0AHh9TdVt+TpcxyGPhVbUJh8NZut+w4cRIKu2xGoZZGDeLgv9ijXvDPrCdBxtpvQnF99C0bCTdPM7MK1FSCQEaRmNUgzEj0H7R+gmWi0T4HUD8ZY7y0vWbYPqT3Z07My1u2BfRBYpXnv1655/b777TkSlzSasORhSTT0AkW25u90Rsj2JWr39+49/Ia9uOyFtZIirugEv2v7v/l0grN linuxkidd@zenbook", "key_options": null, "keyfile": "/home/mkidd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkidd", "validate_certs": true} 2020-04-10T06:59:22.618 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2020-04-10T06:59:22.684 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af', u'name': u'jlopez'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrRvNeBa+/GXxkSDmaim7U1wFpdBvgcK9/eP1I4fPS5xAWyxXI0s0ryX4Jwb7auhZdcqPjgUPQvIVVRridKJPz7l6jUlbIqdnZu4VHuaGG/DfIVxsSmw5aI5qI9Fg/s16JPwOUuzcLHsSSh8tFxY9L1yvs3ztQnKHW6nN5+zhPbW8UjmTNUksXXAeCM/OZn+IOOzzvfh0vVe/mghIW2JpOjHkZv4qwaWYcHg6+wWE4m8l84VIegDBiONknZXcQaRj82nHtfdnb2FSiV4+gKP18i90ai0iTKKXUkMJNFKp1T5MuF7UuJLA7l10FmgQJsZBS2Loa5oeqbHYeztNAfUxF JCLO@oh-mbp-jcl", "key_options": null, "keyfile": "/home/jlopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlopez", "validate_certs": true} 2020-04-10T06:59:22.780 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n', u'name': u'dfuller'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/r/Xf58szqJ3wLdTE+mRK1Z11f/+VABmx6804NO77ncdC5ZvD3Kx6WP/oje01iwWEVXNAgEOaXp6j+U10aPvBMaS30qf/IhUcrXwO1YcGzZ0pV6UZMxcTuJw6w1VXy/YtgW5BlIfSm78zabztES9BqCgjYYM2rml3L1teDF4WU1QiI97iW0R57yc0t/gqXGpxqSFQ+iNYAl0Q1MGUgYltLvPrhm9WI2WhR2/GivdVMEM5lViNlegxneAZRb9YZMCG2kmSP+9Zd2NR367OrlgzmgeY/9u+KGXQiyfaO330s+iksxVwwE3fDl+KnqXuHSJNccJV7iXzZKej2QAire4X dfuller@Douglass-MacBook-Pro.local", "key_options": null, "keyfile": "/home/dfuller/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dfuller", "validate_certs": true} 2020-04-10T06:59:22.871 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb', u'name': u'haomaiwang'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDFkUz8Wr2RPiY8+xf2hL4qMT+jA4mo1gmpIF8q7uVR03vX5NhzQVxVnXiyLCFv6susSMxpBafa7aLutqigXhDGJWUbsJII5yTnFYVer0QmudEsLmoDyvI1HCNk1IVY6aIESE50N3Ndx+agWasIU8TGKqJdDMPyiUtfrvsrGWi/91ZLX6aJSrFqBJtTpxDoNe4XL6QRPNv0wb8GF0az5graXkzVe3GkaQS9lyw8m2ougDvW1SLvoeE+lhUUGhJGkkCZKv1f+QNLhUHyQ3XkXqh4MrP3pPUKIuQ7nKiWDjCP6Db+ZcejjTRC19D2gLjhrw3Z2J7nn/loijlFzV5dp+BL haomaiwang@gmail.com", "key_options": null, "keyfile": "/home/haomaiwang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "haomaiwang", "validate_certs": true} 2020-04-10T06:59:22.937 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n', u'name': u'owasserm'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDH/YT1mGZg50qDdngGQ4Ea+jfJGZ582J6pQbYkrvuGsyATMzNwk73sAbn7mgbpw7nkW/ol/vwahXHZ5heC056BFnqiZ7tjoDeotDx/1XxFtkbyIaVflUeD1vBww38v6IslLHcVhPilauPVZSfVzEHI7JB0a+9OwNrEH3+dt1B6vR+WRy2gHG/aEJffPMv4yv/0FgRWy3cEiUvGlLq0KVVTbsqqLQhw/u5k8bBIoibBtZLFAMTrjLYp9Dv8dxJROvmey6O2TuX6gCY1rLGTAArYULpXH73OLIqaksJKc3srgoJrTSZPO6bo3BQeLz4ODJ7uvTpvQIhacBpp/dq1ln7p owasserm@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCiqChxNdielMPBchs0u6bxWstqU9oeY2K9qf1PNwEIrwDYTrpDjsdoVnS750tOBjWY0wr9Gf9ATOJXi89151oao9GaUueQ1rTgmu7GtPlKsuBU 2020-04-10T06:59:22.938 INFO:teuthology.task.ansible.out:VNMFzLEa0mqneqDA4AfknisFt/LnL8fLM4BddT25tmQ4oUGS7F17p4kXqwboK/DsJciCMWWc/5AT4NVwdnYFzIA6ta25/bB94kpktaZ6HDw7G8q/mK62Yza3EZfq8SP7p0sPL7oVsInQ9AutcVonmALgzPa9t1/MPGaXUIqARaWtwe4MSlNEIs4afLFyteBVeq+dgU01yeyjFivwVmuS+aPFTm6sjGGe8/gOsFDIJiVertOqi663m7q1WYGlVANfif7uL2+uJ9yrvOmrFh5s/CJMJBRG1SLfK1j/ivFwvPAQc/uMqEIRzrVZOsuNLPO4zHPWzmBAPUteb+2BqSdUFYMlBxb3lmhFFffroDqnOK1nbdAfmyKiosUizQgX2Gs2wjjtejuhS9dpYkNu+sBOujsxa2mZUouyWf4fXyWGVjthrCZaM+FyeSDpJrgwgRiSKNB8aRxB10ZNo9WZEy6eH2adp0Xf6v2KsKvryDndqBEkBI8zLbGrPVHGSEqiSBWc+NeAB06uSscJd0pHU3hDg0qW0EaxOXaial9qrBiwcFVKEr0g7LDURQ6KZsZy9w== owasserm@owasserm.redhat.com", "key_options": null, "keyfile": "/home/owasserm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "owasserm", "validate_certs": true} 2020-04-10T06:59:23.034 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608', u'name': u'jdillaman'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA296in+8Sr9slF8hrzAV3dNz0q94Q4FlFaMYw08II5PQm6HOpabGOGOwOud/PPeh3lMUHhs45N5WnbgdfQdzz9lpDoNBcqxeACdMicdECMHtYWf2GuFUdTAnPBFVl8wFW5JcziQYJePUcAUJRXGGfdMLMppAi8VEuhMw/z3jz89UN+zf51Y9eT3rgAVSCR+Zn47kite4OB2S1aVMEcZBE0fPnh60i+rpN26s80fh3wG2N9JrPBEvst6LbL0aEixrlRUtoFHZuWNCIjntVMANa+idpuWSxN2dzfcoxpGLF1TFzqx0moznp0PnDs7ZAFFS1/EkeSCgBflyhPV6csG3VNw== jdillaman@ceph1.example.com", "key_options": null, "keyfile": "/home/jdillaman/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jdillaman", "validate_certs": true} 2020-04-10T06:59:23.082 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n', u'name': u'vasu'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVFy1sjGvvHwS7Q6E61ykH4dUGfxW9+a7Xo3JMpTBT33/E8bTrYF6ZHSdH+uxScisV9Vll222NJCLXyx2s9EOcv4L88vWDTZGCaUb3iA5dweLm7EYi1NXT/p+jWDA7MAF1FQGSAQafTHpNRxLcrVPgsLDy35weV4xj7ShEU+w7EBDb75jlJXqy0luJLKNBrLzz/wDV9Oe6y3APwQBhtO7vGrVzrWEF/LWLe7U9ad3DQjvLM7+H+QVpNy+nt7us73K8vHDk1KrPlCCjrcN+fSGPD/ILlxrDrDeqmmBCtPMa7PSSjCyAV/EW0fsmArcHrHXKeKWTKXG9sDL0KBqos1kB vasu@vasuSrv\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQConeaJ2kvUOMy+jeKvXZTw655L3KK0ZG0HONIySFrRNkGa/M/hYf8oMNVh9SEax7vLNme0kwk7d71EhgtMGNsvHtQOkshWxM9P0XPk4HgB/OL+2LQ/7M3SwD4vuXPbqvrQS63YMoxcHBue5Ol5hXmM+qSoZajpUrfB2woqbp7aKsCpoK4Cgj4XfLHfh0n239ae38DU6GoAcL0GzMsxMdQhWzBwOO+ZHfowohc1pfY0+xGIMW1OlMN1eO7kuK2gGS/YRVOnLw9uu/2NErIjBzCihNN0PYWur2JE+KmU1x6VMaky49cKmAlc5WUqZ+FpV3gvnb3cLjBClnCkdjIcLeRR vasukulkarni@ovpn-116-227.phx2.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRFyOO7qitrq6Hk2NSl28h4laRTxA2zezy1wwjTqs6UL+oZjNRYQjeWD3dDNPZrUmc9JyI3j1EN11cCsKAp8YXz3ItB46M0BPeJ4TXCpg36vxs7/twSDMq/EoXA5sRBbUJFf+bthcNemwG1WsJoxRVzMezbWBTHvz5i4Hbh4sL3PBbac3FhT/3b5Q4tKdO6bXGXmNLAoALyVZZMfoSz9CWzvrO+g8Houb9iJiaXHp293y+ND9Wicl9wHPckO1VcocZ+ldByujgX1cz6m+pxmf6 2020-04-10T06:59:23.084 INFO:teuthology.task.ansible.out:AKY1OMJj5PjmpN0HrgpV2QLVA+ba9qBiDwi1g8sRHXkOETMZ2jq/2a/mmTtvrsF vasu@teuthology", "key_options": null, "keyfile": "/home/vasu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vasu", "validate_certs": true} 2020-04-10T06:59:23.181 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3', u'name': u'kchai'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAuTvkaUtEaLbLcYA9Zw3MoHI5ZPudKf6CAbAZdhp4RMVujimR61TT6LdRF20g6F7pH7GrI/1WP2bu7a3sZyCZ1nCJCOhrTTS+2wwy7oGo/0yY4l+Bw6i7LGmJgqpO5dDxwADMlaHUwJ7Rf1Q+Pd7NmggkAVTmaN6qUGffz9hZsj7ibD/Z8/GFtQsCEMloTLtnz4NB6Gxe54UXEKz9pvjpXBhs0xWsJtDKaPFrbKRKswuH45XPuwJhGr249ZYtwQqcgXSWhOUcPTwf5Y6fj27aL2dx566DxQzqzNbcTcO96MguIy8Ufnc0kIOQQw9oHJZtg82zwM6+2mWr9J/K94Gv9w== kefu@save.node2", "key_options": null, "keyfile": "/home/kchai/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchai", "validate_certs": true} 2020-04-10T06:59:23.236 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37', u'name': u'smohan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlOImwCV0AGu5L29ABaefV9Vqjv43WYJYFuLTxGQQdeQGcways6iNKGIX/nzf3UWkwgg25OVSpzB7zf6FjBx+HcQU+RJ7FLzeaz0s8ZxBXqWwtOYOvP9cXmAXA07hD5UAfxZKC1twBzL82bKJ2ZSmzfQmBjvpUOqRs7XwsQIClOW/ycY+jLgLRLFFDxzADsgnn6UEGlHeMETYocUbe89P3wbY5fHhqZmn6e5ncuAQMsiLRa9d+QhSdpD07Y/P65EzqRF4UOX+zpOc6gPlgHFir9pvf0bTbbBG/qKN9+KGqzZMRJGKenpHM6EMfLkf0QmrT4s4zd4ZnUQPq3HN3pul3 redhat@unused\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9n4ege98BNmvgUxogdcpptMILFfEisAQaiLUvSKpTsoy2gF7C6xSJXgDaziT5dW4ad2El4xmKQ/e6/e14Vx5i9pGbJz4NvL36Xmb3ent2PtY18i6d3CbnmGI8J7YH4YS9QvR7FC/KuvwLR6AdJ+uQShGhqG8kWJMjbqrtaxu8l1hq8ymBXw3O1Ht+xrZ/r3dEkAIVF3bMCSUnZpEbSsNoeiydEQl0ZCw+meXYmSw15vYVtKpYscZxqADqhSJJxIEHZA4E88dq6aYdV2AZ/s4cJK+tBl/ckBnclI3VGHSGn6oaHoO/fCHtKiENplDuKoLGQPO6xJ2Q/apHSGD7McY3 redhat@network\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDn/XgNfvqytLHgfBUdQIhJiWBlj+JrXqGZYvAZZJIWs32Ugm1/lCQTssGZrEj3+Qv81jLfkP6VQ8gXSWJkzUK4csxRxkZKXqcYLDE1csdmAFDOfWUFJwhMsCiiCyBiW9oapioLpvd40nw8k9qM4QuzaUhVonoGXMDDL1/dCgkbIjiQxxTJPHDEJFjtzul+VuhlhHqE6S1syoJG+UKo3Q1tOBCs8rwaPXEfCXFsyv+ZmcEv1+mpM8QRjdr1D1hfUfwb4CHtaa2AE6B5wL+zoY6+jsrTwZSwCfeIdgEZ8e3V2iXP5F7G6eXotu5r7mGYJClHZFLSfbNPN5J+EbRORl9/ wbhack3r@dhcp35-24.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHYkp1nuRSYSAl6me1xuoTU30DW2YpAR6EH1c2adUC9tNtTZbMnab11aitQLAtZb9SKsRgnoB5KpH+zJ0VNKCuJU227r9vZwcVUruSGwA6X7LOrSkK0gFqNRZkVoKhC9oGHSU0zegYBDxx8VLxSV3CE56b+KEosEo6iJe7TkXy3wXHK6am0uhQPEdSHtdUdD683CGIffdF1RFmAFnzTPn96lC9rLe1mIGO+jVQx4zIo3i33RFQLy++4DtQw6Lrgp26Xr85HmQekjgyiya0VkR9Ypy7fgF2ucBGegnOgZDGCCY4G2Ruq 2020-04-10T06:59:23.238 INFO:teuthology.task.ansible.out:iZyKz2TZaOecDwrLkbYM7EGp8up5uLFRZsV shmohan@shmohan.remote.csb", "key_options": null, "keyfile": "/home/smohan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smohan", "validate_certs": true} 2020-04-10T06:59:23.333 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2020-04-10T06:59:23.380 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114', u'name': u'abhi'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZePxJmYerihzA0aQbMt7K7bctPEDcwPHiO8EVuJowp6zwhSmtDnYuFybKJsee4l8+rDKR6T5Ugdl5pLjSzQ+Ba00OSH7+qWC6shvVDYju5mIooJnc+kwx8RgapbGtDdl/yYskBb1KD5g/3+i/2aM+Igk+7nQR7Cny5+WnUEvHd7PHBbUXQSuu+mlUlnXCVMM8g8qlc1I3WQL5eQoIqGjSWRgZUIsn3P5oryYBzbuG9jqLwbaohJvZRJqkimQDe7zCFk6bB3z39pyOWUN+sldRvyL8FPyEmqog4IwpjcYyCuCnPhnb6fNq9yB65RUIde82Kkzx3LZrYOXTrdHgD/lr abhi@trusty", "key_options": null, "keyfile": "/home/abhi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "abhi", "validate_certs": true} 2020-04-10T06:59:23.497 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n', u'name': u'dfuller'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/r/Xf58szqJ3wLdTE+mRK1Z11f/+VABmx6804NO77ncdC5ZvD3Kx6WP/oje01iwWEVXNAgEOaXp6j+U10aPvBMaS30qf/IhUcrXwO1YcGzZ0pV6UZMxcTuJw6w1VXy/YtgW5BlIfSm78zabztES9BqCgjYYM2rml3L1teDF4WU1QiI97iW0R57yc0t/gqXGpxqSFQ+iNYAl0Q1MGUgYltLvPrhm9WI2WhR2/GivdVMEM5lViNlegxneAZRb9YZMCG2kmSP+9Zd2NR367OrlgzmgeY/9u+KGXQiyfaO330s+iksxVwwE3fDl+KnqXuHSJNccJV7iXzZKej2QAire4X dfuller@Douglass-MacBook-Pro.local", "key_options": null, "keyfile": "/home/dfuller/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dfuller", "validate_certs": true} 2020-04-10T06:59:23.519 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611', u'name': u'cbodley'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDcsBe6CzgWh64iZAH8zbU6ra4yEiUBSsiUn0AHzyQurvjeRpL2Mut6B1WEi0kJG7BAM6b13JrhnD9BW9yIMIUIPpVTa2o4zaXP90+bvjGrgDeJbxv3T9/TGh13k+WWyampxm2lo90/+Paql/HObQyyIv7Wan0DOdIe2/Ue4ooWH0tu3kj+3unJTuw82oEDEn+9tGs0oleG2c8XN0gQyVl1DzR2Gqf9PCzXK7hInC3niY2huoSAZzVFEgsniKEWqTnVWYi49X2SsJhUyHCj+mqrTG3YnW30nha3EXOWIbQGblXVoTIOc+gNreqdGByOMICKSnQmPSG4DAl3c1Y4h4A71T3Q043IUv7v7S9tAEKq2h6oJ9I/a5z0u4kaL/+B7Cw7l4bitTdcZ0rcqSvOkKUNNgxujbucMSSc4Tsh5L0762xaziD9nQZl5bteb3VtsMAueqhu7L3sJy69+HGg7oF68t5TguErXfI2UNVGOn3eRj+zN/zSwLUXMoROlpM5yHXSzHvpsbgZiHBImM2CeG4qaPtqzFw7VH6NWJv4ypb0htOMWsbXmLshNTca1KXrrBcVozSJC8wKgEOxu4spk26G8x1IVTsi2ypyYX+irtAqZrrpIPdQ0Ss9NjKPhFNXAxJgLHj6TlH8UNh1gRfN/QTyzpFSe4EH6uOlnWuxcuM8ZQ== cbodley@redhat.com", "key_options": null, "keyfile": "/home/cbodley/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cbodley", "validate_certs": true} 2020-04-10T06:59:23.641 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n', u'name': u'owasserm'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDH/YT1mGZg50qDdngGQ4Ea+jfJGZ582J6pQbYkrvuGsyATMzNwk73sAbn7mgbpw7nkW/ol/vwahXHZ5heC056BFnqiZ7tjoDeotDx/1XxFtkbyIaVflUeD1vBww38v6IslLHcVhPilauPVZSfVzEHI7JB0a+9OwNrEH3+dt1B6vR+WRy2gHG/aEJffPMv4yv/0FgRWy3cEiUvGlLq0KVVTbsqqLQhw/u5k8bBIoibBtZLFAMTrjLYp9Dv8dxJROvmey6O2TuX6gCY1rLGTAArYULpXH73OLIqaksJKc3srgoJrTSZPO6bo3BQeLz4ODJ7uvTpvQIhacBpp/dq1ln7p owasserm@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCiqChxNdielMPBchs0u6bxWstqU9oeY2K9qf1PNwEIrwDYTrpDjsdoVnS750tOBjWY0wr9Gf9ATOJXi89151oao9GaUueQ1rTgmu7GtPlKsuBUVNMFzLEa0mqneqDA4AfknisFt/LnL8fLM4BddT25tmQ4oUGS7F17p4kXqwboK/DsJciCMWWc/5AT4NVwdnYFzIA6ta25/bB94kpktaZ6HDw7G8q/mK62Yza3EZfq8SP7p0sPL7oVsInQ9AutcVonmALgzPa9t1/MPGaXUIqARaWtwe4MSlNEIs4afLFyteBVeq+dgU01yeyjFivwVmuS+aPFTm6sjGGe8/gOsFDIJiVertOqi663m7q1WYGlVANfif7uL2+uJ9yrvOmrFh5s/CJMJBRG1SLfK1j/ivFwvPAQc/uMqEIRzrVZOsuNLPO4zHPWzmBAPUteb+2BqSdUFYMlBxb3lmhFFffroDqnOK1nbdAfmyKiosUizQgX2Gs2wjjtejuhS9dpYkNu+sBOujsxa2mZUouyWf4fXyWGVjthrCZaM+FyeSDpJrgwgRiSKNB8aRxB10ZNo9WZEy6eH2adp0Xf6v2KsKvryDndqBEkBI8zLbGrPVHGSEqiSBWc+NeAB06uSscJd0pHU3hDg0qW0EaxOXaial9qrBiwcFVKEr0g7LDURQ6KZsZy9w== owasserm@owasserm.redhat.com", "key_options": null, "keyfile": "/home/owasserm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "owasserm", "validate_certs": true} 2020-04-10T06:59:23.681 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195', u'name': u'mwatts'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrtHyRuOSEXAM6mwL4qoM+NM4hpIKhaRS+/Y6RDoFKszkXFw/jC9ZYh7q39rQpWQBKHVe4RyPN9yyGWydFwaR/BjZi14ippmYOqVG4f7b+k39NFbMJRcSxYC+NvkqxK7+zpsnIWC+RdJ+blZi+zCbaaAYcMDSvBZW19FsJSLrO7hh4R1cuzgV+Xrmwdc1/VH+HqPMu3tHGv7N0SfsklWI8Inr0aiL83QqqgKPcVxljSWZGYvvFyTQOrkNQeaLURJNER9WIIac9o57WOINw8kR+ojebcOUBsrxZH9j+gMjsJ752Rph03wdMUpFcnS6yfvI4Ial7x/sGms4FE+wL/vVF mdw@pika", "key_options": null, "keyfile": "/home/mwatts/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mwatts", "validate_certs": true} 2020-04-10T06:59:23.807 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n', u'name': u'vasu'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVFy1sjGvvHwS7Q6E61ykH4dUGfxW9+a7Xo3JMpTBT33/E8bTrYF6ZHSdH+uxScisV9Vll222NJCLXyx2s9EOcv4L88vWDTZGCaUb3iA5dweLm7EYi1NXT/p+jWDA7MAF1FQGSAQafTHpNRxLcrVPgsLDy35weV4xj7ShEU+w7EBDb75jlJXqy0luJLKNBrLzz/wDV9Oe6y3APwQBhtO7vGrVzrWEF/LWLe7U9ad3DQjvLM7+H+QVpNy+nt7us73K8vHDk1KrPlCCjrcN+fSGPD/ILlxrDrDeqmmBCtPMa7PSSjCyAV/EW0fsmArcHrHXKeKWTKXG9sDL0KBqos1kB vasu@vasuSrv\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQConeaJ2kvUOMy+jeKvXZTw655L3KK0ZG0HONIySFrRNkGa/M/hYf8oMNVh9SEax7vLNme0kwk7d71EhgtMGNsvHtQOkshWxM9P0XPk4HgB/OL+2LQ/7M3SwD4vuXPbqvrQS63YMoxcHBue5Ol5hXmM+qSoZajpUrfB2woqbp7aKsCpoK4Cgj4XfLHfh0n239ae38DU6GoAcL0GzMsxMdQhWzBwOO+ZHfowohc1pfY0+xGIMW1OlMN1eO7kuK2gGS/YRVOnLw9uu/2NErIjBzCihNN0PYWur2JE+KmU1x6VMaky49cKmAlc5WUqZ+FpV3gvnb3cLjBClnCkdjIcLeRR vasukulkarni@ovpn-116-227.phx2.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRFyOO7qitrq6Hk2NSl28h4laRTxA2zezy1wwjTqs6UL+oZjNRYQjeWD3dDNPZrUmc9JyI3j1EN11cCsKAp8YXz3ItB46M0BPeJ4TXCpg36vxs7/twSDMq/EoXA5sRBbUJFf+bthcNemwG1WsJoxRVzMezbWBTHvz5i4Hbh4sL3PBbac3FhT/3b5Q4tKdO6bXGXmNLAoALyVZZMfoSz9CWzvrO+g8Houb9iJiaXHp293y+ND9Wicl9wHPckO1VcocZ+ldByujgX1cz6m+pxmf6 2020-04-10T06:59:23.809 INFO:teuthology.task.ansible.out:AKY1OMJj5PjmpN0HrgpV2QLVA+ba9qBiDwi1g8sRHXkOETMZ2jq/2a/mmTtvrsF vasu@teuthology", "key_options": null, "keyfile": "/home/vasu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vasu", "validate_certs": true} ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg 2020-04-10T06:59:23.810 INFO:teuthology.task.ansible.out:9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2020-04-10T06:59:23.967 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37', u'name': u'smohan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlOImwCV0AGu5L29ABaefV9Vqjv43WYJYFuLTxGQQdeQGcways6iNKGIX/nzf3UWkwgg25OVSpzB7zf6FjBx+HcQU+RJ7FLzeaz0s8ZxBXqWwtOYOvP9cXmAXA07hD5UAfxZKC1twBzL82bKJ2ZSmzfQmBjvpUOqRs7XwsQIClOW/ycY+jLgLRLFFDxzADsgnn6UEGlHeMETYocUbe89P3wbY5fHhqZmn6e5ncuAQMsiLRa9d+QhSdpD07Y/P65EzqRF4UOX+zpOc6gPlgHFir9pvf0bTbbBG/qKN9+KGqzZMRJGKenpHM6EMfLkf0QmrT4s4zd4ZnUQPq3HN3pul3 redhat@unused\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9n4ege98BNmvgUxogdcpptMILFfEisAQaiLUvSKpTsoy2gF7C6xSJXgDaziT5dW4ad2El4xmKQ/e6/e14Vx5i9pGbJz4NvL36Xmb3ent2PtY18i6d3CbnmGI8J7YH4YS9QvR7FC/KuvwLR6AdJ+uQShGhqG8kWJMjbqrtaxu8l1hq8ymBXw3O1Ht+xrZ/r3dEkAIVF3bMCSUnZpEbSsNoeiydEQl0ZCw+meXYmSw15vYVtKpYscZxqADqhSJJxIEHZA4E88dq6aYdV2AZ/s4cJK+tBl/ckBnclI3VGHSGn6oaHoO/fCHtKiENplDuKoLGQPO6xJ2Q/apHSGD7McY3 redhat@network\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDn/XgNfvqytLHgfBUdQIhJiWBlj+JrXqGZYvAZZJIWs32Ugm1/lCQTssGZrEj3+Qv81jLfkP6VQ8gXSWJkzUK4csxRxkZKXqcYLDE1csdmAFDOfWUFJwhMsCiiCyBiW9oapioLpvd40nw8k9qM4QuzaUhVonoGXMDDL1/dCgkbIjiQxxTJPHDEJFjtzul+VuhlhHqE6S1syoJG+UKo3Q1tOBCs8rwaPXEfCXFsyv+ZmcEv1+mpM8QRjdr1D1hfUfwb4CHtaa2AE6B5wL+zoY6+jsrTwZSwCfeIdgEZ8e3V2iXP5F7G6eXotu5r7mGYJClHZFLSfbNPN5J+EbRORl9/ wbhack3r@dhcp35-24.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHYkp1nuRSYSAl6me1xuoTU30DW2YpAR6EH1c2adUC9tNtTZbMnab11aitQLAtZb9SKsRgnoB5KpH+zJ0VNKCuJU227r9vZwcVUruSGwA6X7LOrSkK0gFqNRZkVoKhC9oGHSU0zegYBDxx8VLxSV3CE56b+KEosEo6iJe7TkXy3wXHK6am0uhQPEdSHtdUdD683CGIffdF1RFmAFnzTPn96lC9rLe1mIGO+jVQx4zIo3i33RFQLy++4DtQw6Lrgp26Xr85HmQekjgyiya0VkR9Ypy7fgF2ucBGegnOgZDGCCY4G2Ruq 2020-04-10T06:59:23.969 INFO:teuthology.task.ansible.out:iZyKz2TZaOecDwrLkbYM7EGp8up5uLFRZsV shmohan@shmohan.remote.csb", "key_options": null, "keyfile": "/home/smohan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smohan", "validate_certs": true} 2020-04-10T06:59:23.979 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea', u'name': u'pdonnell'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQD3vdP9QHyNFUNPOdplezEbdYsQfcC91ncATI6Fl/xHifmUbcvjIRqZHkCY6FNWUpMeg9S3+wlHJtzjVqJJ/MlQML3EtwnzQkeWv8bjtuZ07OqULYIv0zEx63XUfSnrdFleJ8+KDSpn7krkYE44q85B88IL+ZBeq5eCXFJ1TtmsDcvsN8WqZNWMdUGqgH3Bb4D8+YKrEfGxoievyxZ1A1DnBHan2l8foTKYggRrnuSbTb59cFpxY3R7Arc5Cn9kDJT1nDVBA5SSGCfRMiNyBOUzQVYe/UjKTaCpngglXC3wYnjoGvGcpksHLQCxyuJPB6djC1tdF+3Jehdf8y8+2TBzWXiiPq2XwS58sER5GcLLLQv26pPL8hgu6CdI0sHxgcgQBgpjoLwUMHmv13CnN/AAcZY4RRDK2X5zd6wFSEhuuCoUqh/m7K219IxI6fVkLaApC851D/ua/pj9OMjiOaJ+D13cPrgfqxva4eK8FQZvqQm7aUK/TT216QNYmn7hkzNjMTkjv/L5yYQ5Ym92AHy44wHzLPrtDB5mk94vHOjG+KlAt9ofuLncMSAXWe+fOqxuIQ6VfbL2LUv0EGYhHe4XOVj98q5utsVQxQbXAZzNQaq9FWXs2A4xcVP7aWj0Fooi17LN/W0ng4Iijir5hyLr7k7M1G6resgiY7p83biNow== pdonnell@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQCpgLVt7Zphmf6nqD+9BC0N91jdmaY6ZpvrYWlzbhxXITQ6juIEtegnlZ+8ohdeEfHFBiyu1xOSQSM1X1ugvpPQE94UqQg7JVb0tuoUCNJHYnIuv8T4YgyP0/vEV48k2G0BBWlP+IJ5f1iDvZuLHWLR5JSgGqh5ELaaHr7Cbyb6JlImldb7SDIoPHMCZ+n6dLO8l5Zy7qWelw6aeA770xmPucBr6t/BVns2g77XgJz6zFd99yEaDy27i8jfzL7WqR9i93yVE9EsL7tuOjg6G9KUzxBF0bg9iVsJXypvcBuG0JwQ+SDKqYw/cN4iLy5qBa6Siqrfpc/LKU21VNmJ0e5eXNgKKbMBpBb8GVmkoLaSRcXw7LclWopjNDA+meUhDAwU7b97N7cqN8BYhBKQyiShXFHMKLuNo2fSuTRt/icfb9VisKNGBdvwgV0oOlrqbXMzZKmj9amb9b+3JDBTGB7ioCsj5x/Evf9srXRrQ9kikIr1aBbVXIWir/lasNWVydcmUYsNi8L7W0vXArLwp+GgOXJFeZUiD3z9LG4CouRxMSwoj3lgWeL2ZR8feH4D/SP/+QmBdU6+aG97au9qreeXi5X8QTR3LND0DwwEKczX+3s+Fa950Oieom/gXC57PVKVh90KVoAGw3HzJTlZrauOtLkYbTUAX6woOMhAtJ5oWuDDT4UrHQj37jRrrl2dETLwKwUJN2+2L07vnnOZ4ABcGWLsU6d3bXeUBlw3pdsYqPebDKUEpjL2kUOw+z 2020-04-10T06:59:23.980 INFO:teuthology.task.ansible.out:mfjEfdVx9BfrvExvDX+3MkxccCJ8FY7od1s5fGEFHsXnIiMnLO/HZb/JN+Aoe+/ExadIcYwJnWcCJyly2/532kGFUlzOliCwPxgJpjWCKQAv6LjWmxoPXiA4JI0bp6y7lMEjmV2Bp3mwphlsW/r8Ie5hg1UPh/ttHRp/VZgcV8e6nNmP/qc6k9b2DWTNFqtWqm1hXLOwq08HzUEeH9mTqvenyAbv2RFMOS18Y2K+4BKUzHwZujrz9qEh/kqtpFScEIXkOJ+HZz3mI6XfjWFIPS487yRGfUvW3+nSWhvHc8vJY21bFRKdjD84qCVlZ2BfOCGUZSKftVItEiqXMTRs3RZd5epfCVZbJuA0hG16jUseeHqqZqDtlbLkzkxtS6v3iGRnPh/Ke/94FHHCzvYY8cmMYAVaM6p/G3edWftbaEtN70Fo7VKxPrpPzuSRZ7Vxmnb+Zad95DxxAC/vtMy/vkBVIP6dyn4HGxqa1oECFMe4oXdhnxZQoPzvWxb8upcXeAeOMMbcIlMq5a+dCBPGFNKhNhrb+9uAEDHwjE7rZlMJVSvt9v1S4hll6jiYIap+7n4reSKrS45aO9U0xiKXJJDohh pdonnell@icewind\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQC8MdUJzIvTCn0fM+hv1HAO/Lwjy/C/V7stQRWGIxyrRHbniyX8iuaU3j7YNFglSQPJq+YRM9UnGauM9/0J9tHKLsMp+zRQYrtxU/ThiTUN8TRJY11mGUol4yLsqkTtm6oq5sl6o+LfuA4hzaYc3XLrFHRJuCUgbtPG71Q0C/y3U+itrDK4wXUxBa4YzZzcIMg+c88VXwoouPbiL0gW5PCXzMaNtKlOqBUKlE935MLn70wIEfN/EwAAbss0ga1moqeqmi9Nlvm20DNIVFmkAGtYoq6CHXtL/uehzai5Sueop8Vmd8+1U8gDTo6JpiagdD8ZNf0zAbRx587KAT5ZMnahJpO276j7RNa85ZFkNkEnBLqBHRkCkTCYq1GehYyRaR69uuKyczFiIoTfAQgPh7VlnpT9525b/4nFsBDOkWxxWAIJR61YiKemPV/n/e9SwhvVsMDb2B/RBriwR7Mi2SZ/lBHGB+z0OA+EE2rGZy4LFVOfUczqIdpltlNotXan4M4F361i+bX19KE+f2RLySn3+GiFv0CdLwslS/y4xysz+FWxw6B0Y/EkpiDU3CdWM61+THGGm/DVhfL/Pvca7k8PonHFYw/x5w1nBcADpkM9JF+m5eOnp+weTEzl58sS+3GnoAuL8YFJwMW1RmE50xM0o5E3SDc8XftUvaSZz8mhII3uwDDukbELfmkLbnIeipQKyjO+GEx0JBp4Itz82qGuao0Wi32U9wHzhSWnX4GQ0pKZnq9frSlyCNwHu/iMiKF8sA2bX3dvplFJ9nKWPefcU8bdl/upr5hYrd2GttjUSkWZa+UuF264yx3ztcN7jotRhEmmdEcHVzKQ26/NyRI4/9DQpDKY7zw4O2wFkXWg5L9kodXwWzgy4cyOr4dMD9LBVLOpHc0CAji5/+lfuYQbu1mwVdfOzCXDqByjt7zCbeN7KuUh403sJ8PTQVp642xA9i4AkPv5g+fqi5C0++1DbZP0lmLfKtTswHhoN+kAqZu2N4Mu6mNVNKyZgW5h1pZzNSGWYcZttyYmMCHhPketReL0LPJ+VUsqQCVohuta3aaSe44InA28GZsXzMj5B0el/84bnE7cN4b+rfGD25NT73JEt3RgPa9W9jqT1hwd/nf53BDKpp7DlHjR77WSWT4h4F7bhbM1RsGSqAfZjCUJT6Rq+IuM2XcrLDY8zIeArkqaZLXMzfGj6KJ7cQes05P5LYfY4osXiq68Gs/7gzF31PQRRGOYAoqMN7752sn0ZMcpnUj3mJutQcncRUN4+naTMjU36WWD7it0YuQBnb2IJ4iKjYLInw8pr2zKgWtgr1TSCM4xKp/v1/ZMmbB 2020-04-10T06:59:23.982 INFO:teuthology.task.ansible.out:mS+dTzNC7Mpkt8vdCNjzZ5KZ5 pdonnell@aglarond", "key_options": null, "keyfile": "/home/pdonnell/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdonnell", "validate_certs": true} 2020-04-10T06:59:24.107 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114', u'name': u'abhi'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZePxJmYerihzA0aQbMt7K7bctPEDcwPHiO8EVuJowp6zwhSmtDnYuFybKJsee4l8+rDKR6T5Ugdl5pLjSzQ+Ba00OSH7+qWC6shvVDYju5mIooJnc+kwx8RgapbGtDdl/yYskBb1KD5g/3+i/2aM+Igk+7nQR7Cny5+WnUEvHd7PHBbUXQSuu+mlUlnXCVMM8g8qlc1I3WQL5eQoIqGjSWRgZUIsn3P5oryYBzbuG9jqLwbaohJvZRJqkimQDe7zCFk6bB3z39pyOWUN+sldRvyL8FPyEmqog4IwpjcYyCuCnPhnb6fNq9yB65RUIde82Kkzx3LZrYOXTrdHgD/lr abhi@trusty", "key_options": null, "keyfile": "/home/abhi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "abhi", "validate_certs": true} 2020-04-10T06:59:24.150 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868', u'name': u'jlayton'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDKzhyWb3PmU88m8BbgpEuBzYM6+x28z6aZwHSesiyfXMfF4Y7IZjtKaTxH0ph/O9GQCPhRCVhRd8TlgKWJGhts1E+VkQfHfT18CgmrPWzj3gifc2FJhL97vB1Pd3jksGnOPAikgYPHP46SVMTtyvXCkOUuo7E29WrkdpjawcqM1rfpw63OnzHf7Ar7Sqh/othB7ajyVQ92t+XWSEqj7K+NRfTaq5L5RBWbwarM885o8xdI5mimOZ0QjzKOhFDORrI+HEVlkutyB8E20nZgo7TUy+kYeLVhEqb3A6Pi2QbA9GFLT69kA/H9nebhj7Cuim9yrx4OKeZuIncG0UxSQF3QmWflAZHBs8L/jniTz4DJ5id5jR29tvGK2LeddCEMB1L2cHwWHlG0aX5jaNEJe+nG6H5Tcg3+M1RPM8twBkRQmS3UzAyX/MTH1kktyFUHn8CdBXe0WTuoFSvXLB6BCHWzDYZOt0tIV/xMtBwhy+XGYA4oZSsDU5m7VlSVLBGL8DbRX69Z8AGey2y3FoFu/roY6NmDhTxUedJ2ZfoKSwdZD6jjatdO1621nonBnJxrDgaXCWzT7HtnWcc3GwxZoBrqkvgpsx88f4lNi22OpYVPV6A4hECe04u6OJnHSWVk3ft3p5PaLNF8aEXbiIgzZKAwX3LxpvEDFH/t1sdihhpmjw== jlayton@synchrony.poochiereds.net", "key_options": null, "keyfile": "/home/jlayton/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlayton", "validate_certs": true} 2020-04-10T06:59:24.245 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611', u'name': u'cbodley'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDcsBe6CzgWh64iZAH8zbU6ra4yEiUBSsiUn0AHzyQurvjeRpL2Mut6B1WEi0kJG7BAM6b13JrhnD9BW9yIMIUIPpVTa2o4zaXP90+bvjGrgDeJbxv3T9/TGh13k+WWyampxm2lo90/+Paql/HObQyyIv7Wan0DOdIe2/Ue4ooWH0tu3kj+3unJTuw82oEDEn+9tGs0oleG2c8XN0gQyVl1DzR2Gqf9PCzXK7hInC3niY2huoSAZzVFEgsniKEWqTnVWYi49X2SsJhUyHCj+mqrTG3YnW30nha3EXOWIbQGblXVoTIOc+gNreqdGByOMICKSnQmPSG4DAl3c1Y4h4A71T3Q043IUv7v7S9tAEKq2h6oJ9I/a5z0u4kaL/+B7Cw7l4bitTdcZ0rcqSvOkKUNNgxujbucMSSc4Tsh5L0762xaziD9nQZl5bteb3VtsMAueqhu7L3sJy69+HGg7oF68t5TguErXfI2UNVGOn3eRj+zN/zSwLUXMoROlpM5yHXSzHvpsbgZiHBImM2CeG4qaPtqzFw7VH6NWJv4ypb0htOMWsbXmLshNTca1KXrrBcVozSJC8wKgEOxu4spk26G8x1IVTsi2ypyYX+irtAqZrrpIPdQ0Ss9NjKPhFNXAxJgLHj6TlH8UNh1gRfN/QTyzpFSe4EH6uOlnWuxcuM8ZQ== cbodley@redhat.com", "key_options": null, "keyfile": "/home/cbodley/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cbodley", "validate_certs": true} 2020-04-10T06:59:24.269 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f', u'name': u'rzarzynski'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCijPApnekNl/dsY+fleKYstEzNkKflBEcCoKtjItUCDJCVLVFOsDBrV94EGvDWpr13aj9sSwpUzRbeu/vOW8ZT853HH6Dqg2+eoYwG6vDRprZb93nw6l4vtU59/M0yKkyi9CPp2+YUo31cCFzYWpq8bv2j1xBruDSUCUdn9nJrHiLFqA4QEwSpw+uVCjSKzQL+LjlKmE8Plfd+8mNCQeY/OFsjxAztwN+4M50SZCnOVCIwEXoYRNd82JOXMOaqZJazSWTO8Tml6+Nu0hyEcvHkHxWRDSM+6LmXNnjlVUPyDN2h1/VAYATpOZ2VhmRarDoftCp9Wzi7eQdL/CGiMyuD rzarzyns@redhat.com", "key_options": null, "keyfile": "/home/rzarzynski/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rzarzynski", "validate_certs": true} 2020-04-10T06:59:24.404 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195', u'name': u'mwatts'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrtHyRuOSEXAM6mwL4qoM+NM4hpIKhaRS+/Y6RDoFKszkXFw/jC9ZYh7q39rQpWQBKHVe4RyPN9yyGWydFwaR/BjZi14ippmYOqVG4f7b+k39NFbMJRcSxYC+NvkqxK7+zpsnIWC+RdJ+blZi+zCbaaAYcMDSvBZW19FsJSLrO7hh4R1cuzgV+Xrmwdc1/VH+HqPMu3tHGv7N0SfsklWI8Inr0aiL83QqqgKPcVxljSWZGYvvFyTQOrkNQeaLURJNER9WIIac9o57WOINw8kR+ojebcOUBsrxZH9j+gMjsJ752Rph03wdMUpFcnS6yfvI4Ial7x/sGms4FE+wL/vVF mdw@pika", "key_options": null, "keyfile": "/home/mwatts/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mwatts", "validate_certs": true} 2020-04-10T06:59:24.419 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e', u'name': u'rdias'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdGthG4qx90VQ8mJ7TI/IFFzptkSqBHYZsFIVu06IQgHbnFBvjao6heGt4UlkWJkegPCDWpKVAe7wFuXDpPlj/VssRHB5LBnYHKGfkNAzMNiMjpEZsTdy+9w39CCujipz0AlOSfrUdANBZ11AnG9VgQSPEukX5487juob55lFM6J3x+v4opBF3rSlux3+thbr8V0w4oCHMD9/aHvVjFoBnxo4oirR4jc/jgC2fQCMiAJhFKJGhjkKFiNHxcB3npBuRnpy+59BcScOAzSXdJ7DGnwxQLjymnhqsAcWinjjJhOjaq6NaoeKv5I7zQLaVGaQdIyuuAK1Dnb7trpfQU7mR rdias@rdias-suse-laptop", "key_options": null, "keyfile": "/home/rdias/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rdias", "validate_certs": true} 2020-04-10T06:59:24.535 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options" 2020-04-10T06:59:24.537 INFO:teuthology.task.ansible.out:: null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2020-04-10T06:59:24.562 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953', u'name': u'vshankar'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEEjb7yrcY6I6XbMU0hCps1NqBuYmA6HDmMcS/l3i5wJRBitIXYVlv6pefUG9gdBYR5NneccLfVFURu+DC63MJa2ELJvVXsQ6YUOX39bJUT9KlZs8mH+nlTD7tlPmP17Ae3k8sT0H42Q+/5+b5YvdAU+03aPhzntmy38SyvLKrvowRk63wHgcE9p23LY9tY3+HSJu86cb/IdePk1GLjnrMeT9aM+5j2eETrmBjMArP+/2UIsT5eb8MBkQzJ299SCC2gfV/qIVkorFl9xdejctAwNwdT0SPTMxCHxaiK3wZ0e/5HHTbSgR5YlxOlKYq93XBVYIaFKPCePk1At1a3Pjh cephalopod@octopus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDcrTZoKO5e5xs2PB2UuAruvGlJESTHNAtqipbC8yJtRg8WGVDLt//rGu36QNSbpbJYMZgqgKsXBYolJB1ckqLYLv5j8RAoVwbW4QqKaV95y7x0bbXs/YAPnRBhEO1nfpT8GHQusUZvTbz38FDaQwstsYoncmsyZwBrqus9ycFavpY4wfpKoWtzeiK5Wjq7ervThOs98ZJKXJXOOxeI7mDakE3Jv7q2tBNu/rU1kzxIl65kWURzgxox6EmIZXQK3FzKt0jjdKtApVYSlx4ODLPI4iyxOv+MbRe3Nnj6FDhfz8/FeP/U5wWxdYx2WslzZCOI1BJ0VDLu3xEpLlu4vPzp vshankar@h3ckers-pride", "key_options": null, "keyfile": "/home/vshankar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vshankar", "validate_certs": true} 2020-04-10T06:59:24.679 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea', u'name': u'pdonnell'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQD3vdP9QHyNFUNPOdplezEbdYsQfcC91ncATI6Fl/xHifmUbcvjIRqZHkCY6FNWUpMeg9S3+wlHJtzjVqJJ/MlQML3EtwnzQkeWv8bjtuZ07OqULYIv0zEx63XUfSnrdFleJ8+KDSpn7krkYE44q85B88IL+ZBeq5eCXFJ1TtmsDcvsN8WqZNWMdUGqgH3Bb4D8+YKrEfGxoievyxZ1A1DnBHan2l8foTKYggRrnuSbTb59cFpxY3R7Arc5Cn9kDJT1nDVBA5SSGCfRMiNyBOUzQVYe/UjKTaCpngglXC3wYnjoGvGcpksHLQCxyuJPB6djC1tdF+3Jehdf8y8+2TBzWXiiPq2XwS58sER5GcLLLQv26pPL8hgu6CdI0sHxgcgQBgpjoLwUMHmv13CnN/AAcZY4RRDK2X5zd6wFSEhuuCoUqh/m7K219IxI6fVkLaApC851D/ua/pj9OMjiOaJ+D13cPrgfqxva4eK8FQZvqQm7aUK/TT216QNYmn7hkzNjMTkjv/L5yYQ5Ym92AHy44wHzLPrtDB5mk94vHOjG+KlAt9ofuLncMSAXWe+fOqxuIQ6VfbL2LUv0EGYhHe4XOVj98q5utsVQxQbXAZzNQaq9FWXs2A4xcVP7aWj0Fooi17LN/W0ng4Iijir5hyLr7k7M1G6resgiY7p83biNow== pdonnell@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQCpgLVt7Zphmf6nqD+9BC0N91jdmaY6ZpvrYWlzbhxXITQ6juIEtegnlZ+8ohdeEfHFBiyu1xOSQSM1X1ugvpPQE94UqQg7JVb0tuoUCNJHYnIuv8T4YgyP0/vEV48k2G0BBWlP+IJ5f1iDvZuLHWLR5JSgGqh5ELaaHr7Cbyb6JlImldb7SDIoPHMCZ+n6dLO8l5Zy7qWelw6aeA770xmPucBr6t/BVns2g77XgJz6zFd99yEaDy27i8jfzL7WqR9i93yVE9EsL7tuOjg6G9KUzxBF0bg9iVsJXypvcBuG0JwQ+SDKqYw/cN4iLy5qBa6Siqrfpc/LKU21VNmJ0e5eXNgKKbMBpBb8GVmkoLaSRcXw7LclWopjNDA+meUhDAwU7b97N7cqN8BYhBKQyiShXFHMKLuNo2fSuTRt/icfb9VisKNGBdvwgV0oOlrqbXMzZKmj9amb9b+3JDBTGB7ioCsj5x/Evf9srXRrQ9kikIr1aBbVXIWir/lasNWVydcmUYsNi8L7W0vXArLwp+GgOXJFeZUiD3z9LG4CouRxMSwoj3lgWeL2ZR8feH4D/SP/+QmBdU6+aG97au9qreeXi5X8QTR3LND0DwwEKczX+3s+Fa950Oieom/gXC57PVKVh90KVoAGw3HzJTlZrauOtLkYbTUAX6woOMhAtJ5oWuDDT4UrHQj37jRrrl2dETLwKwUJN2+2L07vnnOZ4ABcGWLsU6d3bXeUBlw3pdsYqPebDKUEpjL2kUOw+z 2020-04-10T06:59:24.680 INFO:teuthology.task.ansible.out:mfjEfdVx9BfrvExvDX+3MkxccCJ8FY7od1s5fGEFHsXnIiMnLO/HZb/JN+Aoe+/ExadIcYwJnWcCJyly2/532kGFUlzOliCwPxgJpjWCKQAv6LjWmxoPXiA4JI0bp6y7lMEjmV2Bp3mwphlsW/r8Ie5hg1UPh/ttHRp/VZgcV8e6nNmP/qc6k9b2DWTNFqtWqm1hXLOwq08HzUEeH9mTqvenyAbv2RFMOS18Y2K+4BKUzHwZujrz9qEh/kqtpFScEIXkOJ+HZz3mI6XfjWFIPS487yRGfUvW3+nSWhvHc8vJY21bFRKdjD84qCVlZ2BfOCGUZSKftVItEiqXMTRs3RZd5epfCVZbJuA0hG16jUseeHqqZqDtlbLkzkxtS6v3iGRnPh/Ke/94FHHCzvYY8cmMYAVaM6p/G3edWftbaEtN70Fo7VKxPrpPzuSRZ7Vxmnb+Zad95DxxAC/vtMy/vkBVIP6dyn4HGxqa1oECFMe4oXdhnxZQoPzvWxb8upcXeAeOMMbcIlMq5a+dCBPGFNKhNhrb+9uAEDHwjE7rZlMJVSvt9v1S4hll6jiYIap+7n4reSKrS45aO9U0xiKXJJDohh pdonnell@icewind\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQC8MdUJzIvTCn0fM+hv1HAO/Lwjy/C/V7stQRWGIxyrRHbniyX8iuaU3j7YNFglSQPJq+YRM9UnGauM9/0J9tHKLsMp+zRQYrtxU/ThiTUN8TRJY11mGUol4yLsqkTtm6oq5sl6o+LfuA4hzaYc3XLrFHRJuCUgbtPG71Q0C/y3U+itrDK4wXUxBa4YzZzcIMg+c88VXwoouPbiL0gW5PCXzMaNtKlOqBUKlE935MLn70wIEfN/EwAAbss0ga1moqeqmi9Nlvm20DNIVFmkAGtYoq6CHXtL/uehzai5Sueop8Vmd8+1U8gDTo6JpiagdD8ZNf0zAbRx587KAT5ZMnahJpO276j7RNa85ZFkNkEnBLqBHRkCkTCYq1GehYyRaR69uuKyczFiIoTfAQgPh7VlnpT9525b/4nFsBDOkWxxWAIJR61YiKemPV/n/e9SwhvVsMDb2B/RBriwR7Mi2SZ/lBHGB+z0OA+EE2rGZy4LFVOfUczqIdpltlNotXan4M4F361i+bX19KE+f2RLySn3+GiFv0CdLwslS/y4xysz+FWxw6B0Y/EkpiDU3CdWM61+THGGm/DVhfL/Pvca7k8PonHFYw/x5w1nBcADpkM9JF+m5eOnp+weTEzl58sS+3GnoAuL8YFJwMW1RmE50xM0o5E3SDc8XftUvaSZz8mhII3uwDDukbELfmkLbnIeipQKyjO+GEx0JBp4Itz82qGuao0Wi32U9wHzhSWnX4GQ0pKZnq9frSlyCNwHu/iMiKF8sA2bX3dvplFJ9nKWPefcU8bdl/upr5hYrd2GttjUSkWZa+UuF264yx3ztcN7jotRhEmmdEcHVzKQ26/NyRI4/9DQpDKY7zw4O2wFkXWg5L9kodXwWzgy4cyOr4dMD9LBVLOpHc0CAji5/+lfuYQbu1mwVdfOzCXDqByjt7zCbeN7KuUh403sJ8PTQVp642xA9i4AkPv5g+fqi5C0++1DbZP0lmLfKtTswHhoN+kAqZu2N4Mu6mNVNKyZgW5h1pZzNSGWYcZttyYmMCHhPketReL0LPJ+VUsqQCVohuta3aaSe44InA28GZsXzMj5B0el/84bnE7cN4b+rfGD25NT73JEt3RgPa9W9jqT1hwd/nf53BDKpp7DlHjR77WSWT4h4F7bhbM1RsGSqAfZjCUJT6Rq+IuM2XcrLDY8zIeArkqaZLXMzfGj6KJ7cQes05P5LYfY4osXiq68Gs/7gzF31PQRRGOYAoqMN7752sn0ZMcpnUj3mJutQcncRUN4+naTMjU36WWD7it0YuQBnb2IJ4iKjYLInw8pr2zKgWtgr1TSCM4xKp/v1/ZMmbB 2020-04-10T06:59:24.681 INFO:teuthology.task.ansible.out:mS+dTzNC7Mpkt8vdCNjzZ5KZ5 pdonnell@aglarond", "key_options": null, "keyfile": "/home/pdonnell/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdonnell", "validate_certs": true} 2020-04-10T06:59:24.691 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58', u'name': u'akupczyk'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+CtnvyVRSkIvD15O/q7/vQmD4oE8q7Mo6oT3kqTXaEaQ7iDayC2biUC0PC4OztPlNJxPUcKK5DfWw4F79OKeheqUbZRhsCC3Ge2JGoXHmsf0kBYdioMGiAxm+f8M/v1KESSSxvEnU7o+oi6VtFchh4Dl/WZ54rqZc/oQrlGNTHulFAuyoIAlKkCe3N0GdHN54PB+26QbLXvOuqpWmjVFbcjgRC62mkZl3LuCBhviXS4Rl5/ZE0QIHubCwV5XVrsqvscLS64bi06W8RN2vL56Vn3N9TGV4d2dlbE1PaBBESoTQ7Q0iUN32OAZLo7nF/1fiPY3Xc8r35juya5k+KR9z adam@TP50", "key_options": null, "keyfile": "/home/akupczyk/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akupczyk", "validate_certs": true} 2020-04-10T06:59:24.857 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868', u'name': u'jlayton'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDKzhyWb3PmU88m8BbgpEuBzYM6+x28z6aZwHSesiyfXMfF4Y7IZjtKaTxH0ph/O9GQCPhRCVhRd8TlgKWJGhts1E+VkQfHfT18CgmrPWzj3gifc2FJhL97vB1Pd3jksGnOPAikgYPHP46SVMTtyvXCkOUuo7E29WrkdpjawcqM1rfpw63OnzHf7Ar7Sqh/othB7ajyVQ92t+XWSEqj7K+NRfTaq5L5RBWbwarM885o8xdI5mimOZ0QjzKOhFDORrI+HEVlkutyB8E20nZgo7TUy+kYeLVhEqb3A6Pi2QbA9GFLT69kA/H9nebhj7Cuim9yrx4OKeZuIncG0UxSQF3QmWflAZHBs8L/jniTz4DJ5id5jR29tvGK2LeddCEMB1L2cHwWHlG0aX5jaNEJe+nG6H5Tcg3+M1RPM8twBkRQmS3UzAyX/MTH1kktyFUHn8CdBXe0WTuoFSvXLB6BCHWzDYZOt0tIV/xMtBwhy+XGYA4oZSsDU5m7VlSVLBGL8DbRX69Z8AGey2y3FoFu/roY6NmDhTxUedJ2ZfoKSwdZD6jjatdO1621nonBnJxrDgaXCWzT7HtnWcc3GwxZoBrqkvgpsx88f4lNi22OpYVPV6A4hECe04u6OJnHSWVk3ft3p5PaLNF8aEXbiIgzZKAwX3LxpvEDFH/t1sdihhpmjw== jlayton@synchrony.poochiereds.net", "key_options": null, "keyfile": "/home/jlayton/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlayton", "validate_certs": true} 2020-04-10T06:59:24.863 INFO:teuthology.task.ansible.out:changed: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n', u'name': u'nojha'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDeUJcLeyfqIk3ErPoFqh9kOnvUPsk/2hfaYNC2wZLMOSTIs3zXM8xjki8eN6vle0dmmeeAdwF+e2hJ3tCFKsEUQRvMc7ugIBA7L8/DboShYu7ItyQ3QRvy9pi6jnA/tlHA1LHalvyhMw0yZ4kvPbrfNMhZzOKzvL6jX166rqLA5AuiIfdYMBpE5rYq11GVjm0uQKw4UJpBMooNJeovEfQlNBTSV+IJSTiGBpBD7SgsuW3USMeEF3fEgQOcnvYdv5Fbp/crloyLWYc0HTidkLxpjP7vqY7E4BuBJug0zIB7voR3MYq6rAeuWjByPvq6coDlUjnw/QlrvarCbjjSaHdaRXLj/Dj+QqJFviESBr64ZynhHL21Bl6rexW7AwYyPa73Erbus3dN0aVQQ/ApcDTGHT7KRlDQ2YYg05YM4swpVVnrIdV58FzLWkll+ypCK/+kdtk5eRSe/KzZA3s7z4d4WLgolFWs8Fky8Rx1xDGewvMJgFUWVUbcGC13RocbNSxXG80cbIx8L/sclX4lDfGouu2sUXhP19vq1rJ7gZa6XTPohXjPbFS+En2cdlGdKXhppdvjT4QWKNzgQoF7qgWHMqJnMFJ6beT/2zP34KoM3t0ulee04z88VSn8Th8aqvLlGbdrt5LrE90escSaIAR1vNKZ1hCFwGieSKoRtYJTCw== nojha@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6NWn9EzQJoeXkUbaNclYcR9Gga7HFhhXMGxdo/9ypYwlEzQTA6FnEreG3s1EmlQ4hofuc0kMhrjh0YeJZMrETWMoA0SmoX+6iR26UWVtOTk7i1v2TqkTY7llmQneHqQgmZEBXK0jF1HVP5kHY5dhagtcpfsGY5dc0eGUjfqAyh/xrCxciOE/JQ03tDcKIc5eOQIqv46JqnUS90Zr/Cov2qGKKHTa/VO1a2viOmDC/uHi5xXvII1Kh9rHp1oZ2Wm3fiGgT8xZaTJFFhWpdq3AdKfkEaV93yQcCdaUsGBHwDHcPOhxpBZpqZvcCR3I/tigDav4QViB1PRN7JnTIWCIznfrQK3r2Frl6BVFDvydwgRVS43ctEKPOJj8ELMUWX+fpjsKrA9Kn7nUU35ti7qIWKg1Cnz3lzVA29TsEg10E+nT4SVKZtRSWw/2AX0bsfxfSY30yjw5PP+bjnVh/bOOkqR2UkPBj4eeOKMD8nV1an79xP0eH+t36INaJhq6qwxE= nojha@nojha-mac", "key_options": null, "keyfile" 2020-04-10T06:59:24.865 INFO:teuthology.task.ansible.out:: "/home/nojha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nojha", "validate_certs": true} 2020-04-10T06:59:24.981 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43', u'name': u'ifed01'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+c7zV5QmZb0WsgA2UZogoDNaoHcngRVLA8+BYk/U10ZlvCckJSfIXWMlBnqguZgdm/jBwsw5mGVBgHxUSqFNybwIlU00kyPZU6VbyVP+ZNVUf0XIE1NlemldgPBxxcOzSHkHhWrUqA3IKiCllcQVDQzutfUnq+jDT6NkVn4/cyNzNlargt7FR2L8gvskh62jLTBbhEuHMBp3jLCvaHJOk708gXm9qZ5iLOGV0pu7xdg58KE0p8ILClQBTbvoqPgZE9LA3qaOk1NR1n22kuW8JRHaKfD2Hn0Yu/MfKct783Bxb3mxWBKXxmM4otdmsR/R14NH/hIPce9wSkOOnQAC9 ifed@37526", "key_options": null, "keyfile": "/home/ifed01/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ifed01", "validate_certs": true} 2020-04-10T06:59:24.993 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f', u'name': u'rzarzynski'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCijPApnekNl/dsY+fleKYstEzNkKflBEcCoKtjItUCDJCVLVFOsDBrV94EGvDWpr13aj9sSwpUzRbeu/vOW8ZT853HH6Dqg2+eoYwG6vDRprZb93nw6l4vtU59/M0yKkyi9CPp2+YUo31cCFzYWpq8bv2j1xBruDSUCUdn9nJrHiLFqA4QEwSpw+uVCjSKzQL+LjlKmE8Plfd+8mNCQeY/OFsjxAztwN+4M50SZCnOVCIwEXoYRNd82JOXMOaqZJazSWTO8Tml6+Nu0hyEcvHkHxWRDSM+6LmXNnjlVUPyDN2h1/VAYATpOZ2VhmRarDoftCp9Wzi7eQdL/CGiMyuD rzarzyns@redhat.com", "key_options": null, "keyfile": "/home/rzarzynski/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rzarzynski", "validate_certs": true} 2020-04-10T06:59:25.126 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4', u'name': u'myoungwon'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEudQMGBPqfZBV8bXpdQOgVTDP6GFsr5oL4gdbHFMuJCGe0Sfq8LTaSEBQx0MH7tXc7JJ1fU51U7DSt0kfy/PFeV1Y5XnNATiZu/SzdO3jHe06/8CCr3DLnlEmzXASq+tqDJbY1eV011d/TX1jiC8Qd1VNbpdZYExEDsDzwJXnWdh6rjTL4anGjIgQWcmU0XdbUJXvRP4dGjVvZVHZnflAkxLPBbAx/8BBOuBmv6vP4hNs4poi4VLwR6S6Ha1fgMJmsn6StsCVuOvKBJFuGvaulHgHgdoDJ1ryUFikkXXCWoNx4IbwtpQIkSxWMuAovlwwk9G8ozSs+RrFzCavE0f omw@omw-dev", "key_options": null, "keyfile": "/home/myoungwon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "myoungwon", "validate_certs": true} 2020-04-10T06:59:25.143 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e', u'name': u'rdias'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdGthG4qx90VQ8mJ7TI/IFFzptkSqBHYZsFIVu06IQgHbnFBvjao6heGt4UlkWJkegPCDWpKVAe7wFuXDpPlj/VssRHB5LBnYHKGfkNAzMNiMjpEZsTdy+9w39CCujipz0AlOSfrUdANBZ11AnG9VgQSPEukX5487juob55lFM6J3x+v4opBF3rSlux3+thbr8V0w4oCHMD9/aHvVjFoBnxo4oirR4jc/jgC2fQCMiAJhFKJGhjkKFiNHxcB3npBuRnpy+59BcScOAzSXdJ7DGnwxQLjymnhqsAcWinjjJhOjaq6NaoeKv5I7zQLaVGaQdIyuuAK1Dnb7trpfQU7mR rdias@rdias-suse-laptop", "key_options": null, "keyfile": "/home/rdias/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rdias", "validate_certs": true} 2020-04-10T06:59:25.261 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2', u'name': u'gabrioux'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQDZPRW12u7LvaiynB87ad5h/JFtYIVjkKIOym9Y5CAbvatHhZ9fB1WAnYIn2HqiSSPUXT8sFy8w0TmM+aH0/ZGcfXoe0CdTWLtUbXKNlPI5h7TO9rHXESP2iRBpnKs3vK8WnMY0/D5BeESrPfwVScucmzaDrjUqjdZKR/aEeqVvDpQsdaCaoqNg23t4nhQ54BEvK8O+14nQ26Rtmm40QHO1ze+0UK+7nUMMFwxvd1EKjVp21EgYAFEbyQ5Kbbl+fmS/11AGy/yTai0c4qd7ohBzZvx8+sY+c1novG9Q7LhtyxuuYwzYIrGBCkKH5+iC7nt2rGBP+mnTfIgdbJUiXX3cdrMESwOSZy6x6zgbXXME+yfM0myhomKmUkpeZAQInX2KYrnEuPD9kteyK9tZcG6gJSaiUdvlab49Alx5lM2hc62M4s4WKaIgroCoApd8FtNAlTQsdYuycntgNrbPy0LXQ5hw8K8hkvc6RbxISdj/RXWguQSVbJWdS9Pf5NwD3Tq7dsOghJWhx4vBkhW3IECVrPQWVyAmgQ9oj1PoL7/2LZ6h9P8F0KC1qmhm+pWdEPLugOlU6lzQX7RJcukYlsINThM1RGwbPE4h9XcMyHAnODiDF63TDfQ01oKbVXj9XpOGY5eP0EQxKJYjCfWW0Ot6nK2ZCt8OYvoRkWMGnLv11Nub3NNOgh5TB/4nDvEU1cIb/Ikb4BRzVIEbast/pqUJmXe+GQ+ho7mp8y8a+lIxu54BR+VN6isjwODCVt1DYohgixEoul7p5pXvTM+JUY2GD/ddTGwPvVAN66sWnT9bjZZunQwTHqasFCIHNyPhnEnYIIqCo9O1I/D8zRQG3SYNxT6ED9GPARrE/H0TvdOmKaT3e0F2GRJRSToNxol9dpJovUziojOhXyBUjlTqnMY2RVilX90MnzgqR3zGO0aSruTGyNT2Faymqxn6T/50mSdJ8EyWH23BaaXUUW/z3Cjov+yCQjLY48SOWwlxKXw3hGz8xTAxauBYI01ojsVLCpCUGFyKlWS5iFwiLFtr1KPYqeCoS6CoWjiOiFgGe+w3lPPfp2h4rc6auIx2DZ1ByYhWLeHnpkV7sG+q5ckBIRAYmiKyXSGdV4zbXITMWDm+3zu4oj5VPRCfjMlout8OZ4TQm5PUdKZZhSE1jULG9tPDFL7nWwM3z0HET+d9kQYcEO6QB+ma9gZHp1qZJbXvCIC5fQ9y9El++A6N3dBr+ED10Ye88PLoWPeumyAmc24BDdI0skGk89Q0IzoLqagK6HGhPAso8si9wr84CEi4ncpizVLDU2cFKCpef4Sa03HoMbwfTgERtyptKFRbMT+SpuZsnrhF635h9uqFnszowUXV guits@Elisheba", "key_options": null, "keyfile": "/home/gabrioux/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user 2020-04-10T06:59:25.263 INFO:teuthology.task.ansible.out:": "gabrioux", "validate_certs": true} ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953', u'name': u'vshankar'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEEjb7yrcY6I6XbMU0hCps1NqBuYmA6HDmMcS/l3i5wJRBitIXYVlv6pefUG9gdBYR5NneccLfVFURu+DC63MJa2ELJvVXsQ6YUOX39bJUT9KlZs8mH+nlTD7tlPmP17Ae3k8sT0H42Q+/5+b5YvdAU+03aPhzntmy38SyvLKrvowRk63wHgcE9p23LY9tY3+HSJu86cb/IdePk1GLjnrMeT9aM+5j2eETrmBjMArP+/2UIsT5eb8MBkQzJ299SCC2gfV/qIVkorFl9xdejctAwNwdT0SPTMxCHxaiK3wZ0e/5HHTbSgR5YlxOlKYq93XBVYIaFKPCePk1At1a3Pjh cephalopod@octopus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDcrTZoKO5e5xs2PB2UuAruvGlJESTHNAtqipbC8yJtRg8WGVDLt//rGu36QNSbpbJYMZgqgKsXBYolJB1ckqLYLv5j8RAoVwbW4QqKaV95y7x0bbXs/YAPnRBhEO1nfpT8GHQusUZvTbz38FDaQwstsYoncmsyZwBrqus9ycFavpY4wfpKoWtzeiK5Wjq7ervThOs98ZJKXJXOOxeI7mDakE3Jv7q2tBNu/rU1kzxIl65kWURzgxox6EmIZXQK3FzKt0jjdKtApVYSlx4ODLPI4iyxOv+MbRe3Nnj6FDhfz8/FeP/U5wWxdYx2WslzZCOI1BJ0VDLu3xEpLlu4vPzp vshankar@h3ckers-pride", "key_options": null, "keyfile": "/home/vshankar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vshankar", "validate_certs": true} 2020-04-10T06:59:25.416 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f', u'name': u'leseb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpDFqu0Zg94mCjrA0FTB0vMiSBml1pqYCDKHhKBrfaza8u3bhtsmls9i+6U/vIwCEVk4GGPNhWduTJK0h9R5MS2eCf3GJcE0XHqRp/8msUCDPOVSZwC7hoYHZKunUIUaIUajOnR+L2slA/a9JaqataFH7zdrKbPdSKBzf5CXYrc0fK+Fz5OOsaI9KITbM0nESVpNCDzRqnA/aarF9MlN79vpTtvwQ+i0z2Km3cxqwV70GPMuFVo4CGf8pCGpbMrIg9OtoSC1svQ2vZzLkH2WrbDqTfYlOEYJJXaiBKcGhGnLZXfLIGSgvHjpv6xHo63yaMihocyg2MwTS8TZ/VtAaH seb@redhat.com", "key_options": null, "keyfile": "/home/leseb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "leseb", "validate_certs": true} 2020-04-10T06:59:25.417 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58', u'name': u'akupczyk'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+CtnvyVRSkIvD15O/q7/vQmD4oE8q7Mo6oT3kqTXaEaQ7iDayC2biUC0PC4OztPlNJxPUcKK5DfWw4F79OKeheqUbZRhsCC3Ge2JGoXHmsf0kBYdioMGiAxm+f8M/v1KESSSxvEnU7o+oi6VtFchh4Dl/WZ54rqZc/oQrlGNTHulFAuyoIAlKkCe3N0GdHN54PB+26QbLXvOuqpWmjVFbcjgRC62mkZl3LuCBhviXS4Rl5/ZE0QIHubCwV5XVrsqvscLS64bi06W8RN2vL56Vn3N9TGV4d2dlbE1PaBBESoTQ7Q0iUN32OAZLo7nF/1fiPY3Xc8r35juya5k+KR9z adam@TP50", "key_options": null, "keyfile": "/home/akupczyk/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akupczyk", "validate_certs": true} 2020-04-10T06:59:25.558 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca', u'name': u'hchen'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzjJ6nlPn/OXP3knQ6orVId2QLswql9gRetKYlicKDBrdFg0MES03EIuAonu0aUR4NEReR7P1iB8eNnezrHyAu5mxIJlZp0rz7ljcBcTLL+11IJU9I9OjBFwzXGqJ1KK4B4Qfw35oc2O6F5V0JP2nHhHgyny37udJLImhxHy3RNmDwc86JhEBCVR3ouYiZuxgxxbA9BXc6+Lkdu9hEMMCDlaEXwNmMyASVU9Wf3dKnMTSCdar2PANquWbpNQZ5cgoTm39xgust5CEyYJ53aOdrti0DYK7qyzxUXre6ckZGrH4oGv2XrDaxpACIiHXt+slvow51gsZnBRHeayMAxQzGQ== hchen@hchen.csb", "key_options": null, "keyfile": "/home/hchen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hchen", "validate_certs": true} 2020-04-10T06:59:25.570 INFO:teuthology.task.ansible.out:changed: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n', u'name': u'nojha'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDeUJcLeyfqIk3ErPoFqh9kOnvUPsk/2hfaYNC2wZLMOSTIs3zXM8xjki8eN6vle0dmmeeAdwF+e2hJ3tCFKsEUQRvMc7ugIBA7L8/DboShYu7ItyQ3QRvy9pi6jnA/tlHA1LHalvyhMw0yZ4kvPbrfNMhZzOKzvL6jX166rqLA5AuiIfdYMBpE5rYq11GVjm0uQKw4UJpBMooNJeovEfQlNBTSV+IJSTiGBpBD7SgsuW3USMeEF3fEgQOcnvYdv5Fbp/crloyLWYc0HTidkLxpjP7vqY7E4BuBJug0zIB7voR3MYq6rAeuWjByPvq6coDlUjnw/QlrvarCbjjSaHdaRXLj/Dj+QqJFviESBr64ZynhHL21Bl6rexW7AwYyPa73Erbus3dN0aVQQ/ApcDTGHT7KRlDQ2YYg05YM4swpVVnrIdV58FzLWkll+ypCK/+kdtk5eRSe/KzZA3s7z4d4WLgolFWs8Fky8Rx1xDGewvMJgFUWVUbcGC13RocbNSxXG80cbIx8L/sclX4lDfGouu2sUXhP19vq1rJ7gZa6XTPohXjPbFS+En2cdlGdKXhppdvjT4QWKNzgQoF7qgWHMqJnMFJ6beT/2zP34KoM3t0ulee04z88VSn8Th8aqvLlGbdrt5LrE90escSaIAR1vNKZ1hCFwGieSKoRtYJTCw== nojha@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6NWn9EzQJoeXkUbaNclYcR9Gga7HFhhXMGxdo/9ypYwlEzQTA6FnEreG3s1EmlQ4hofuc0kMhrjh0YeJZMrETWMoA0SmoX+6iR26UWVtOTk7i1v2TqkTY7llmQneHqQgmZEBXK0jF1HVP5kHY5dhagtcpfsGY5dc0eGUjfqAyh/xrCxciOE/JQ03tDcKIc5eOQIqv46JqnUS90Zr/Cov2qGKKHTa/VO1a2viOmDC/uHi5xXvII1Kh9rHp1oZ2Wm3fiGgT8xZaTJFFhWpdq3AdKfkEaV93yQcCdaUsGBHwDHcPOhxpBZpqZvcCR3I/tigDav4QViB1PRN7JnTIWCIznfrQK3r2Frl6BVFDvydwgRVS43ctEKPOJj8ELMUWX+fpjsKrA9Kn7nUU35ti7qIWKg1Cnz3lzVA29TsEg10E+nT4SVKZtRSWw/2AX0bsfxfSY30yjw5PP+bjnVh/bOOkqR2UkPBj4eeOKMD8nV1an79xP0eH+t36INaJhq6qwxE= nojha@nojha-mac", "key_options": null, "keyfile" 2020-04-10T06:59:25.571 INFO:teuthology.task.ansible.out:: "/home/nojha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nojha", "validate_certs": true} 2020-04-10T06:59:25.701 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43', u'name': u'ifed01'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+c7zV5QmZb0WsgA2UZogoDNaoHcngRVLA8+BYk/U10ZlvCckJSfIXWMlBnqguZgdm/jBwsw5mGVBgHxUSqFNybwIlU00kyPZU6VbyVP+ZNVUf0XIE1NlemldgPBxxcOzSHkHhWrUqA3IKiCllcQVDQzutfUnq+jDT6NkVn4/cyNzNlargt7FR2L8gvskh62jLTBbhEuHMBp3jLCvaHJOk708gXm9qZ5iLOGV0pu7xdg58KE0p8ILClQBTbvoqPgZE9LA3qaOk1NR1n22kuW8JRHaKfD2Hn0Yu/MfKct783Bxb3mxWBKXxmM4otdmsR/R14NH/hIPce9wSkOOnQAC9 ifed@37526", "key_options": null, "keyfile": "/home/ifed01/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ifed01", "validate_certs": true} 2020-04-10T06:59:25.715 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n', u'name': u'jcollin'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCqZKc+/7iPLhapA/7SYmLxwyKEM95D6eFfsDBNKbEUey/B1OlqYSo+yKB/LdFDyUcd7shMDnhFFg47lNMANrsZdtE4iMPVAa1YhcrTgNbV3HSp9cthKixvU/gkt9Fnb6PEckRzr19wEwxtoNg3Fh/R6YCiSjPwqHmArtC1G2hzk560EJpozx6xylMR5IafatNDoZVw2YQ35fZvHbhHD6meQELducHJ1bKEEHxVIRSTcmXDpBz/vAvBoQOEPuH6PmC/oSwL9HVVIsazf65F4QfK98d+8NCK9TemgXyGDk3Sjz6xdm76yBhFzDQEAzO12U1YtS7Q7zGVsjmSDav2dd7 jcollin@earth\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC89dHSsgjgtIjJ3e/pcMthliQlX2vtdXL7pk62nD64230Fm1PfOVCnzn4LqDhBbGgJT+BrlwOPfzStpLgnUT3l67S834SJzhvSTTpQp2pznnQg1rZOiXjvNhbQwVYV4rphXWqay8weWvFIb5QfXE66mYA1etZtcOb1SMY2ofAjUiA8bAQS5aHpzcMVs4N1A9x5GJx/7/Bj1h8zPpSXikDYasJc3++UMGc45C/AAf725YwijK7xMnZNXv8uTBoGzO/Ko24R3VdvtfMVn4Hi4/FwuRd5bRYcJXR/nUdtA3IinqvlnTpFV2LbfgVTHiUMGhvewRTB+8S8ij4nnAlko3H3bhqZZTk/y7d4K/IerY0mozVIc2x8rMLfqhF9OKv1QFB9fskRZcF/ENHITnmLqCJPg9rgmyv1gCQ5Tbds1OI+IYQIowdiwtaViiTSIy7/CedbHqMz6aDnPsJFMs24lcUXe2d8uZ26RqMinMFGXg0zF7tifuVmxj7zG8DT7MFORDm5f/Hhkng05R5XZesStTBxpEU0MGuzWhENt3RuJdstqmY+wkDVW3XwjUvKG/gVYuJYfnBAsRZIZezw2wfvodhc8EmeHELvY1YtB3DGTBXKE1lsKGN7yV5+nl4UtR2IGzBn9GovWVi5ooB8UlSAeUmZix/eAA5teL8ldFrDht7+AQ== jcollin@teuthology\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeApVHu5er+R9h52RLSVYVhXFQSn/nMN38uWoDYPw2bqYBLNEAAMcPl+rnqHubELpXTGXjJYL4n2mjic+2rOLZpDgPLmOOm4PwgYstJjE1KLegKqVzXIa9HfASKMXaHSgwtC6P5vDQkm2MNl8vXuTAITF7VYdDS 2020-04-10T06:59:25.716 INFO:teuthology.task.ansible.out:33Kkke6ViJRZI7szYjbP7z9A1fy26THrvnuBT1Q18yy9vgk/sKo8vjLQfHSLvO90CJzouNEKqmkVpmJEqCPbE+s6Qr4+d2KY7L6Q6GAXxrqXK1jSDrrPsJguE6zZ1oWy/7XFMOr9ssduFWocnjQjPtQj3M+V+7FYAsf5uqQnzeMQTjO6AXYxTI7 jcollin@stratocaster\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDk/DYRp5E8Dla/4XHCF504X20W0u7CijZSCFWT1/5082NgnYeR90Prth3p+4xsw3YC9H5pEAS3XwcbdUb/NtWvZVIAIcMd7NOOEClSIj2Agk7NJxZ/4SfQRLrK3H2Qli/MVnrTxnEa9oeS4zqzv9dKRwFJDUFCoH2CpoumFVAp02uyzk7D/+XqFrXzMJUiSHXfl3YM2rt/vNmCtHtVp04UcG0Kyl2TXY37frRlSOkMMu0zbIC551gOyae6NGYhkYHa7u2SnngXcKM78b8MDvMv3QtuUN5P4WpH5qFRdlKNKsQGrzcQpBDY8AKQ+rX0K1ENhBwer49bihYNMKvjYmvTdLTNWgPCu7dt8JSgJQ8JMMhve0pjvFdpcKDLp0fvXDvJvZdLJM1ID/66kywJspHjShuqQNXRMfd1R9C4F20c+R9LY6eTGwrpLG1DNEez1VBHmOu5hQl03T2PfaThfLfMHRTw8JpGXGLMQarCCTm1PC6Z2Iw7KzorxP1eofGA5q8XoCbwGD01uQRZrwxPq0SHzTAkW2C2M8oEz+k4Mrx7+vGYzSS14hqxrzk91wuYhZCAKuGpqwm1ouJ4wxvWFewjbxLgucMfGSp09zJ2XSYRmtm7SAUifFob6Z9czHGTQerHCOTPuYqzlap2gCCVYtEvQyNB+4IqK/9FtEXTr28x0Q== jcollin@stratocaster", "key_options": null, "keyfile": "/home/jcollin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcollin", "validate_certs": true} 2020-04-10T06:59:25.849 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc', u'name': u'xxg'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdxi9xdXNBYctMHH1cuuyNcPC4I4xVAeD3HZsKH+QwIILiKOTHu/6HZg7Yo1WsuwYSA0MDc+5LCSGu7RsmOEVpcSQ4gtNoFVwsThDyI6yhsm9WQqqXwSPs7ZE+urqd/BDfQ/7oMqJm5bViQMP0vmBExHuIFagj+8f6nspkE2Z7lBbaw8bEbAJZFx74oReOAELsL+Pbtf9Iaqrap4Z3WysmhfP6Rm/q7mncBBmo/fl/bejhThr8fghTPJb81M+XLhMk/rAdVrGdKV9iSt6lLiWdvf0lpTZO4LdIfJSBXr1Xv+lHRtaGXID2ahH7zl/W6dM4baz+hU7ZrbP+vdKg/SVH xxg@xxg", "key_options": null, "keyfile": "/home/xxg/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xxg", "validate_certs": true} 2020-04-10T06:59:25.862 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4', u'name': u'myoungwon'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEudQMGBPqfZBV8bXpdQOgVTDP6GFsr5oL4gdbHFMuJCGe0Sfq8LTaSEBQx0MH7tXc7JJ1fU51U7DSt0kfy/PFeV1Y5XnNATiZu/SzdO3jHe06/8CCr3DLnlEmzXASq+tqDJbY1eV011d/TX1jiC8Qd1VNbpdZYExEDsDzwJXnWdh6rjTL4anGjIgQWcmU0XdbUJXvRP4dGjVvZVHZnflAkxLPBbAx/8BBOuBmv6vP4hNs4poi4VLwR6S6Ha1fgMJmsn6StsCVuOvKBJFuGvaulHgHgdoDJ1ryUFikkXXCWoNx4IbwtpQIkSxWMuAovlwwk9G8ozSs+RrFzCavE0f omw@omw-dev", "key_options": null, "keyfile": "/home/myoungwon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "myoungwon", "validate_certs": true} 2020-04-10T06:59:26.002 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53', u'name': u'pcuzner'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsA63hT3ZAwQ9hsvfY+o2Gy15M1GRnRw1ZBSXEiCs9OJku7T9+LCvvH85Y/7V+mOGcsCZKqVTd5Vy3sHAvAF3M7CYJ+5IIDJKCYslC0wkKcX4jj26aMToAZPms8uvs527BNgKOcvwgIfPjs4WJG06fey5AhsuuL30etbAvbBudxRAKdZSTOsMveneKtXQu3Dx7y0rjfI/RBAhCAvisnMqYFUdajoJIl5gVXl6+8mauUOP5O0mvyvJH2jQveqZXTdsIPyp5mq1nA5psqvivWGz2bEJ+owI0GjfuYbyAi6i+BL4Bsd3GeoAs1G3Dicfr228VIUsT+R3xLBQUj3xIsVAZ pcuzner@redhat.com", "key_options": null, "keyfile": "/home/pcuzner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pcuzner", "validate_certs": true} 2020-04-10T06:59:26.013 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2', u'name': u'gabrioux'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQDZPRW12u7LvaiynB87ad5h/JFtYIVjkKIOym9Y5CAbvatHhZ9fB1WAnYIn2HqiSSPUXT8sFy8w0TmM+aH0/ZGcfXoe0CdTWLtUbXKNlPI5h7TO9rHXESP2iRBpnKs3vK8WnMY0/D5BeESrPfwVScucmzaDrjUqjdZKR/aEeqVvDpQsdaCaoqNg23t4nhQ54BEvK8O+14nQ26Rtmm40QHO1ze+0UK+7nUMMFwxvd1EKjVp21EgYAFEbyQ5Kbbl+fmS/11AGy/yTai0c4qd7ohBzZvx8+sY+c1novG9Q7LhtyxuuYwzYIrGBCkKH5+iC7nt2rGBP+mnTfIgdbJUiXX3cdrMESwOSZy6x6zgbXXME+yfM0myhomKmUkpeZAQInX2KYrnEuPD9kteyK9tZcG6gJSaiUdvlab49Alx5lM2hc62M4s4WKaIgroCoApd8FtNAlTQsdYuycntgNrbPy0LXQ5hw8K8hkvc6RbxISdj/RXWguQSVbJWdS9Pf5NwD3Tq7dsOghJWhx4vBkhW3IECVrPQWVyAmgQ9oj1PoL7/2LZ6h9P8F0KC1qmhm+pWdEPLugOlU6lzQX7RJcukYlsINThM1RGwbPE4h9XcMyHAnODiDF63TDfQ01oKbVXj9XpOGY5eP0EQxKJYjCfWW0Ot6nK2ZCt8OYvoRkWMGnLv11Nub3NNOgh5TB/4nDvEU1cIb/Ikb4BRzVIEbast/pqUJmXe+GQ+ho7mp8y8a+lIxu54BR+VN6isjwODCVt1DYohgixEoul7p5pXvTM+JUY2GD/ddTGwPvVAN66sWnT9bjZZunQwTHqasFCIHNyPhnEnYIIqCo9O1I/D8zRQG3SYNxT6ED9GPARrE/H0TvdOmKaT3e0F2GRJRSToNxol9dpJovUziojOhXyBUjlTqnMY2RVilX90MnzgqR3zGO0aSruTGyNT2Faymqxn6T/50mSdJ8EyWH23BaaXUUW/z3Cjov+yCQjLY48SOWwlxKXw3hGz8xTAxauBYI01ojsVLCpCUGFyKlWS5iFwiLFtr1KPYqeCoS6CoWjiOiFgGe+w3lPPfp2h4rc6auIx2DZ1ByYhWLeHnpkV7sG+q5ckBIRAYmiKyXSGdV4zbXITMWDm+3zu4oj5VPRCfjMlout8OZ4TQm5PUdKZZhSE1jULG9tPDFL7nWwM3z0HET+d9kQYcEO6QB+ma9gZHp1qZJbXvCIC5fQ9y9El++A6N3dBr+ED10Ye88PLoWPeumyAmc24BDdI0skGk89Q0IzoLqagK6HGhPAso8si9wr84CEi4ncpizVLDU2cFKCpef4Sa03HoMbwfTgERtyptKFRbMT+SpuZsnrhF635h9uqFnszowUXV guits@Elisheba", "key_options": null, "keyfile": "/home/gabrioux/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user 2020-04-10T06:59:26.014 INFO:teuthology.task.ansible.out:": "gabrioux", "validate_certs": true} 2020-04-10T06:59:26.133 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2', u'name': u'mkogan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaRX+aBLjb/h12ctmvoMvZro6NrrYDuxROF6QMVpEVucA7jHhwCC/j9dJAq0hOVBO8R3S0bvKQ6bmnCb/4y2J6tq18xKUFCrChNii5OQ+AgM5v1353oZ9SrHN8p9ajvS8w313o2OpXhpaPv3khVDjKZsDmeH8YL/Nx3WQKqIJC1mI4jrI0sZGGJLifEy3pPkO0W1KmguN9AwAWK6+kckAjMa3ElkfK/xuAqBYgNKQUxP9oMgTQRcgpWF8Nzb0npN3v+HOUIRrth1INSchkDKtWob2T+0Zqxwm+Vpi0iFWwqjcN/OVdaDGf34//A3M/tsiQ1g9SqhdhZJCfu5XLmM63 mkogan@mkP50.redhat.com", "key_options": null, "keyfile": "/home/mkogan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkogan", "validate_certs": true} 2020-04-10T06:59:26.197 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f', u'name': u'leseb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpDFqu0Zg94mCjrA0FTB0vMiSBml1pqYCDKHhKBrfaza8u3bhtsmls9i+6U/vIwCEVk4GGPNhWduTJK0h9R5MS2eCf3GJcE0XHqRp/8msUCDPOVSZwC7hoYHZKunUIUaIUajOnR+L2slA/a9JaqataFH7zdrKbPdSKBzf5CXYrc0fK+Fz5OOsaI9KITbM0nESVpNCDzRqnA/aarF9MlN79vpTtvwQ+i0z2Km3cxqwV70GPMuFVo4CGf8pCGpbMrIg9OtoSC1svQ2vZzLkH2WrbDqTfYlOEYJJXaiBKcGhGnLZXfLIGSgvHjpv6xHo63yaMihocyg2MwTS8TZ/VtAaH seb@redhat.com", "key_options": null, "keyfile": "/home/leseb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "leseb", "validate_certs": true} 2020-04-10T06:59:26.272 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2020-04-10T06:59:26.362 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca', u'name': u'hchen'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzjJ6nlPn/OXP3knQ6orVId2QLswql9gRetKYlicKDBrdFg0MES03EIuAonu0aUR4NEReR7P1iB8eNnezrHyAu5mxIJlZp0rz7ljcBcTLL+11IJU9I9OjBFwzXGqJ1KK4B4Qfw35oc2O6F5V0JP2nHhHgyny37udJLImhxHy3RNmDwc86JhEBCVR3ouYiZuxgxxbA9BXc6+Lkdu9hEMMCDlaEXwNmMyASVU9Wf3dKnMTSCdar2PANquWbpNQZ5cgoTm39xgust5CEyYJ53aOdrti0DYK7qyzxUXre6ckZGrH4oGv2XrDaxpACIiHXt+slvow51gsZnBRHeayMAxQzGQ== hchen@hchen.csb", "key_options": null, "keyfile": "/home/hchen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hchen", "validate_certs": true} 2020-04-10T06:59:26.410 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736', u'name': u'henrix'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDZDKRAePFqjugc2ljw4QmM6r7B2dPI39OBB+2HKONtDZI4q+uwgZ1yqnhrbJaU9ux0W5wNzJyp2VfOPhw/9G9jo+efJPg5IySnTonhi1zeCXNpBMY70dJZVCyGEDdWKehvFM9xFBOxBC/h7gOyKqEc28BKoy4cHfULzm4er8PU30MyFBngu8NaoKqyQ3GYAfWOAXF5lUdalbJwl3a+HBb8PjUhbGuznAZF1Ji4JgnNuEZdlNVOKwhEkAFZ1JwXvgLfrEJVQs5b2V9Sp2Vm5ZngsLg7b3RzV5I4B3Lj7QTTraV1zLJGiS5iRdjY7D4W5Uxi2k1X7vaOjySJWWadHyeicNNBhLSfHl6JJ3aMuA8oAAmtgo/8IkJ96GFtgM9j31Wg8wkeTyQrwSeqCuv1uDFr8rp+OvMClhHnDgUklJLWjjYPCDVYR5KV+iOiHL58WJg/b4h6NsoP91wO8P8EcSaiMdq7P77w5nJxDI0KKb//dC8aHtWnRRRlrIS+PNPqKJSNTaSwOTieXvzplhzKFzu2dr79qs+/67iaNE6HBvIX5jVHWCFV3wllyjsC8WYPh4kEVtg2SY00SDOOrMrqANV8DDtHFwwxQTEjz1z5ESuwy9UcUSQfP1/zsAp2OQzPutKs6UipHyBK3pLm6MjaeeaEWu8fmetCqALxYV3h17O+Yw== miguel@hermes.olymp", "key_options": null, "keyfile": "/home/henrix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "henrix", "validate_certs": true} 2020-04-10T06:59:26.484 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n', u'name': u'jcollin'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCqZKc+/7iPLhapA/7SYmLxwyKEM95D6eFfsDBNKbEUey/B1OlqYSo+yKB/LdFDyUcd7shMDnhFFg47lNMANrsZdtE4iMPVAa1YhcrTgNbV3HSp9cthKixvU/gkt9Fnb6PEckRzr19wEwxtoNg3Fh/R6YCiSjPwqHmArtC1G2hzk560EJpozx6xylMR5IafatNDoZVw2YQ35fZvHbhHD6meQELducHJ1bKEEHxVIRSTcmXDpBz/vAvBoQOEPuH6PmC/oSwL9HVVIsazf65F4QfK98d+8NCK9TemgXyGDk3Sjz6xdm76yBhFzDQEAzO12U1YtS7Q7zGVsjmSDav2dd7 jcollin@earth\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC89dHSsgjgtIjJ3e/pcMthliQlX2vtdXL7pk62nD64230Fm1PfOVCnzn4LqDhBbGgJT+BrlwOPfzStpLgnUT3l67S834SJzhvSTTpQp2pznnQg1rZOiXjvNhbQwVYV4rphXWqay8weWvFIb5QfXE66mYA1etZtcOb1SMY2ofAjUiA8bAQS5aHpzcMVs4N1A9x5GJx/7/Bj1h8zPpSXikDYasJc3++UMGc45C/AAf725YwijK7xMnZNXv8uTBoGzO/Ko24R3VdvtfMVn4Hi4/FwuRd5bRYcJXR/nUdtA3IinqvlnTpFV2LbfgVTHiUMGhvewRTB+8S8ij4nnAlko3H3bhqZZTk/y7d4K/IerY0mozVIc2x8rMLfqhF9OKv1QFB9fskRZcF/ENHITnmLqCJPg9rgmyv1gCQ5Tbds1OI+IYQIowdiwtaViiTSIy7/CedbHqMz6aDnPsJFMs24lcUXe2d8uZ26RqMinMFGXg0zF7tifuVmxj7zG8DT7MFORDm5f/Hhkng05R5XZesStTBxpEU0MGuzWhENt3RuJdstqmY+wkDVW3XwjUvKG/gVYuJYfnBAsRZIZezw2wfvodhc8EmeHELvY1YtB3DGTBXKE1lsKGN7yV5+nl4UtR2IGzBn9GovWVi5ooB8UlSAeUmZix/eAA5teL8ldFrDht7+AQ== jcollin@teuthology\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeApVHu5er+R9h52RLSVYVhXFQSn/nMN38uWoDYPw2bqYBLNEAAMcPl+rnqHubELpXTGXjJYL4n2mjic+2rOLZpDgPLmOOm4PwgYstJjE1KLegKqVzXIa9HfASKMXaHSgwtC6P5vDQkm2MNl8vXuTAITF7VYdDS 2020-04-10T06:59:26.486 INFO:teuthology.task.ansible.out:33Kkke6ViJRZI7szYjbP7z9A1fy26THrvnuBT1Q18yy9vgk/sKo8vjLQfHSLvO90CJzouNEKqmkVpmJEqCPbE+s6Qr4+d2KY7L6Q6GAXxrqXK1jSDrrPsJguE6zZ1oWy/7XFMOr9ssduFWocnjQjPtQj3M+V+7FYAsf5uqQnzeMQTjO6AXYxTI7 jcollin@stratocaster\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDk/DYRp5E8Dla/4XHCF504X20W0u7CijZSCFWT1/5082NgnYeR90Prth3p+4xsw3YC9H5pEAS3XwcbdUb/NtWvZVIAIcMd7NOOEClSIj2Agk7NJxZ/4SfQRLrK3H2Qli/MVnrTxnEa9oeS4zqzv9dKRwFJDUFCoH2CpoumFVAp02uyzk7D/+XqFrXzMJUiSHXfl3YM2rt/vNmCtHtVp04UcG0Kyl2TXY37frRlSOkMMu0zbIC551gOyae6NGYhkYHa7u2SnngXcKM78b8MDvMv3QtuUN5P4WpH5qFRdlKNKsQGrzcQpBDY8AKQ+rX0K1ENhBwer49bihYNMKvjYmvTdLTNWgPCu7dt8JSgJQ8JMMhve0pjvFdpcKDLp0fvXDvJvZdLJM1ID/66kywJspHjShuqQNXRMfd1R9C4F20c+R9LY6eTGwrpLG1DNEez1VBHmOu5hQl03T2PfaThfLfMHRTw8JpGXGLMQarCCTm1PC6Z2Iw7KzorxP1eofGA5q8XoCbwGD01uQRZrwxPq0SHzTAkW2C2M8oEz+k4Mrx7+vGYzSS14hqxrzk91wuYhZCAKuGpqwm1ouJ4wxvWFewjbxLgucMfGSp09zJ2XSYRmtm7SAUifFob6Z9czHGTQerHCOTPuYqzlap2gCCVYtEvQyNB+4IqK/9FtEXTr28x0Q== jcollin@stratocaster", "key_options": null, "keyfile": "/home/jcollin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcollin", "validate_certs": true} 2020-04-10T06:59:26.570 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26', u'name': u'pbs1108'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNwmgnIsDDJmsmYzkTAEo3iLj7Jd5Ni2R5vD1Z0NRZ39zJMtbicZ0ec3nAMzFyHrmPnKNFltS1mhehbLbtKNKq1fSo2SMHs11z+N8Nl94F4AY8IhXZ8U9GAorEJFVIxHUNEDqwl55Ov6MfP60UI7rNgy2rUt8aOBmAJVhLEFmJJawJgyPhTrUZAT20OoILphrt81o5bTNfmx++Zs5Dc9VtgIh53hexwKgTvG6fxfKdra9m+J2GXpniOsZjPJq/A0je+t7H3UlxCSu0dRy2Un6l5ZAcoeg74SP50dFgGaxkTQqI6wle6NjtPIdssgrdYTgTcOg9M4jWqAr/5drTUSGh root@centos7-bspark", "key_options": null, "keyfile": "/home/pbs1108/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pbs1108", "validate_certs": true} 2020-04-10T06:59:26.635 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc', u'name': u'xxg'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdxi9xdXNBYctMHH1cuuyNcPC4I4xVAeD3HZsKH+QwIILiKOTHu/6HZg7Yo1WsuwYSA0MDc+5LCSGu7RsmOEVpcSQ4gtNoFVwsThDyI6yhsm9WQqqXwSPs7ZE+urqd/BDfQ/7oMqJm5bViQMP0vmBExHuIFagj+8f6nspkE2Z7lBbaw8bEbAJZFx74oReOAELsL+Pbtf9Iaqrap4Z3WysmhfP6Rm/q7mncBBmo/fl/bejhThr8fghTPJb81M+XLhMk/rAdVrGdKV9iSt6lLiWdvf0lpTZO4LdIfJSBXr1Xv+lHRtaGXID2ahH7zl/W6dM4baz+hU7ZrbP+vdKg/SVH xxg@xxg", "key_options": null, "keyfile": "/home/xxg/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xxg", "validate_certs": true} 2020-04-10T06:59:26.722 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315', u'name': u'clacroix'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDnvVnbK2SNfL8hTmrA09L41RudKW/F7Ty0vvlHjrFjDkdWXVgrafFWGKQZ8dSR6FJ9JLU0X59xiEo4z31ini/Dt2XozFMIr7RdeB3xpiifuUUG5Lgsasd2ezniUoOEukL1a9KkCIxNgmKJuvtlHCt+ZTtyL513yjRICByZbnmBSSpUiett50X10vtvaNBu4LikUJKLc8gP0PHLbaGsU3uOUd655lkFWjWhgExlU8bHWiAZq18Ss63jDlSKFVSk903GKSNcHXMHNPYaXlrYw8NFzejoy/XjxEfmF/Lv3MeAUhHQWx25sh5aDv48mzo8FaIJsL1MrD0QxF1gllobWAfp clacroix@redhat.com", "key_options": null, "keyfile": "/home/clacroix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "clacroix", "validate_certs": true} 2020-04-10T06:59:26.798 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53', u'name': u'pcuzner'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsA63hT3ZAwQ9hsvfY+o2Gy15M1GRnRw1ZBSXEiCs9OJku7T9+LCvvH85Y/7V+mOGcsCZKqVTd5Vy3sHAvAF3M7CYJ+5IIDJKCYslC0wkKcX4jj26aMToAZPms8uvs527BNgKOcvwgIfPjs4WJG06fey5AhsuuL30etbAvbBudxRAKdZSTOsMveneKtXQu3Dx7y0rjfI/RBAhCAvisnMqYFUdajoJIl5gVXl6+8mauUOP5O0mvyvJH2jQveqZXTdsIPyp5mq1nA5psqvivWGz2bEJ+owI0GjfuYbyAi6i+BL4Bsd3GeoAs1G3Dicfr228VIUsT+R3xLBQUj3xIsVAZ pcuzner@redhat.com", "key_options": null, "keyfile": "/home/pcuzner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pcuzner", "validate_certs": true} 2020-04-10T06:59:26.869 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006', u'name': u'epuertat'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAyiGmj+nOAD+il+FvXvZSpUOJQp6hKh6UPaAIQvQZkdeNRWN7wcDXMUlLj+TBFWa1OO+9xqFlnbgVMDR4qy5orUBYlMapUb1rutjbk64XeY58b6uguHwSoMd81QpCCIN4KOk3cldV8UiyX382baoZ9Je2hsOnnBGMsRtHuVHZ90Bj1rIITkwajq//KRs7kjb/qhjk0xkipDIcaSwKjzPlmmnYn69noODwDexpvLYlCF+LowSZEdrOUvVIigSFG7cLTDaU2tLhxSgbvWu8f0pwvjsyW7gOgM+VeRZ6uHC79F1qvgdgQSxflddIgrQpH0rzf6RNxQjR3SBkZ4V1BESa0w== epuertat@epuertat", "key_options": null, "keyfile": "/home/epuertat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "epuertat", "validate_certs": true} 2020-04-10T06:59:26.985 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2', u'name': u'mkogan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaRX+aBLjb/h12ctmvoMvZro6NrrYDuxROF6QMVpEVucA7jHhwCC/j9dJAq0hOVBO8R3S0bvKQ6bmnCb/4y2J6tq18xKUFCrChNii5OQ+AgM5v1353oZ9SrHN8p9ajvS8w313o2OpXhpaPv3khVDjKZsDmeH8YL/Nx3WQKqIJC1mI4jrI0sZGGJLifEy3pPkO0W1KmguN9AwAWK6+kckAjMa3ElkfK/xuAqBYgNKQUxP9oMgTQRcgpWF8Nzb0npN3v+HOUIRrth1INSchkDKtWob2T+0Zqxwm+Vpi0iFWwqjcN/OVdaDGf34//A3M/tsiQ1g9SqhdhZJCfu5XLmM63 mkogan@mkP50.redhat.com", "key_options": null, "keyfile": "/home/mkogan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkogan", "validate_certs": true} 2020-04-10T06:59:27.028 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461', u'name': u'tdehler'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRZyWiDO5JqWHJdUUBr8YReiKXxnL87juFAfkcmw2E9KmANVH2sn3K+OgUCQfBs1Wdg6AbcL2GyMm9alodUruzOm127/bH9OcGmu+8SNsMAYrAF9EzsKJa8Ba/I05rl5a87p1jbYG9Rpo7u3OvLx0LjgqydRYcmjwo/niifYNDTaN0K3v8s1mN4TorUZW/i3XBMBIBjlIiNGs/I7IZreR+1N4V8Nwrp2EWxDv7AwG3nvokL2RTHcsV0KgmAnciv8nXXjIf+uqOPuGA+F2qW/wz92bG8R0O4CjH+QZt4k6UoUIXFRW+LdOVAqmUrb4W0OJ8vsVeukW46SvPnYUTkZnF sonea@think", "key_options": null, "keyfile": "/home/tdehler/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tdehler", "validate_certs": true} 2020-04-10T06:59:27.150 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305', u'name': u'laura'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEBD7CxJJfbLW4jtuLnTwkxVbhCfh/JL3Qj8oeLEkny1tE7QCeXn1bP5bScWSgLH34Wh4TAuH+8/g/koFwcCy1YuoGBdBV4KV7O9MTnFngdqNY6LKzCipm27tnWZEa10hrF8dRFTmhracd6kmmVGHgvGQPVAVONmix+2IZ83wdj8XJSQHeE3PHFLO39tU+ez/T76QaiBLDVo4ctHVt+N9bBbHV6Cr8fyiUUU6YEKMlcedvxGpnMK8oeOSNuDeh+DldcM9KYzNGEOM34jGRIvJp57dxQDQCtPPW5hvBXg7yMyrv6TcJzDd2LFD6AC/Qw9yrEQPXXyBCDkAlCqDR0+rH", "key_options": null, "keyfile": "/home/laura/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "laura", "validate_certs": true} 2020-04-10T06:59:27.151 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2020-04-10T06:59:27.273 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736', u'name': u'henrix'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDZDKRAePFqjugc2ljw4QmM6r7B2dPI39OBB+2HKONtDZI4q+uwgZ1yqnhrbJaU9ux0W5wNzJyp2VfOPhw/9G9jo+efJPg5IySnTonhi1zeCXNpBMY70dJZVCyGEDdWKehvFM9xFBOxBC/h7gOyKqEc28BKoy4cHfULzm4er8PU30MyFBngu8NaoKqyQ3GYAfWOAXF5lUdalbJwl3a+HBb8PjUhbGuznAZF1Ji4JgnNuEZdlNVOKwhEkAFZ1JwXvgLfrEJVQs5b2V9Sp2Vm5ZngsLg7b3RzV5I4B3Lj7QTTraV1zLJGiS5iRdjY7D4W5Uxi2k1X7vaOjySJWWadHyeicNNBhLSfHl6JJ3aMuA8oAAmtgo/8IkJ96GFtgM9j31Wg8wkeTyQrwSeqCuv1uDFr8rp+OvMClhHnDgUklJLWjjYPCDVYR5KV+iOiHL58WJg/b4h6NsoP91wO8P8EcSaiMdq7P77w5nJxDI0KKb//dC8aHtWnRRRlrIS+PNPqKJSNTaSwOTieXvzplhzKFzu2dr79qs+/67iaNE6HBvIX5jVHWCFV3wllyjsC8WYPh4kEVtg2SY00SDOOrMrqANV8DDtHFwwxQTEjz1z5ESuwy9UcUSQfP1/zsAp2OQzPutKs6UipHyBK3pLm6MjaeeaEWu8fmetCqALxYV3h17O+Yw== miguel@hermes.olymp", "key_options": null, "keyfile": "/home/henrix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "henrix", "validate_certs": true} 2020-04-10T06:59:27.285 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce', u'name': u'yaarit'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2k8xsrsLxEqJx3rV6ZZwLH0OjYCDB2uIYzcAyrKEOKlVWO1ICKxkw+gZ9tSiPFNLfq13io9BIrAKJwlrdLCJLmlgCQwpwUQqzmRlRZEm+WmD0dl8LPeQSidyJkPkJgYBEshFPjDHIj10Ei1pJS5svI2l7G1wMWxlt2kHL8Og/4zdFTRctxwqZcvzLmRQcs8lSSpqTkhfeKfmrpR+je8XFbCJJ9m/nJbgyCiZKaGN8nf36pCQDEva4rvaSrK0a6YxL0RWN8W3lqtBPKIehxffr3lYDEkD9EUW5vf6AUKC0ioh4jkPo9WCjaoFsOauMOMVQPHehEAMufEO4Kym6nfrf yaarit@localhost.localdomain", "key_options": null, "keyfile": "/home/yaarit/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yaarit", "validate_certs": true} 2020-04-10T06:59:27.409 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26', u'name': u'pbs1108'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNwmgnIsDDJmsmYzkTAEo3iLj7Jd5Ni2R5vD1Z0NRZ39zJMtbicZ0ec3nAMzFyHrmPnKNFltS1mhehbLbtKNKq1fSo2SMHs11z+N8Nl94F4AY8IhXZ8U9GAorEJFVIxHUNEDqwl55Ov6MfP60UI7rNgy2rUt8aOBmAJVhLEFmJJawJgyPhTrUZAT20OoILphrt81o5bTNfmx++Zs5Dc9VtgIh53hexwKgTvG6fxfKdra9m+J2GXpniOsZjPJq/A0je+t7H3UlxCSu0dRy2Un6l5ZAcoeg74SP50dFgGaxkTQqI6wle6NjtPIdssgrdYTgTcOg9M4jWqAr/5drTUSGh root@centos7-bspark", "key_options": null, "keyfile": "/home/pbs1108/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pbs1108", "validate_certs": true} 2020-04-10T06:59:27.431 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3', u'name': u'rpavani1998'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUznqo3VDmcKRL64P7sbjuxXCW/0ljFTc5zZoZ9UvtZvU2jrRQ2X9GyxcrcQnAWlF5YZHqeTDU5o8ixn5fq1MSmaPRBNDKKVQOfR/vFQV73LiKx4BFsC1/EdaPie/XqltaLDTtfqhxADJlz1H3a3ZlkeVHMROTqo/TidL1YUYlRH4qHxWWBuFuVS+EYfTegSZdjwp+DuJA0iN6X99+DfH2DhiHENjEmlbKN13+H+EPzm0K3ya2w8D6ANodB3AxMn2tGs8mLtG/P4kCjoDnoUVFzKTyUpTtgOjis47uSY4dL8mTbIK9kZAiFxnT5sMClu6KRStDjRIrxbTBQJ8wvlat admin1@rajula", "key_options": null, "keyfile": "/home/rpavani1998/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rpavani1998", "validate_certs": true} 2020-04-10T06:59:27.560 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315', u'name': u'clacroix'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDnvVnbK2SNfL8hTmrA09L41RudKW/F7Ty0vvlHjrFjDkdWXVgrafFWGKQZ8dSR6FJ9JLU0X59xiEo4z31ini/Dt2XozFMIr7RdeB3xpiifuUUG5Lgsasd2ezniUoOEukL1a9KkCIxNgmKJuvtlHCt+ZTtyL513yjRICByZbnmBSSpUiett50X10vtvaNBu4LikUJKLc8gP0PHLbaGsU3uOUd655lkFWjWhgExlU8bHWiAZq18Ss63jDlSKFVSk903GKSNcHXMHNPYaXlrYw8NFzejoy/XjxEfmF/Lv3MeAUhHQWx25sh5aDv48mzo8FaIJsL1MrD0QxF1gllobWAfp clacroix@redhat.com", "key_options": null, "keyfile": "/home/clacroix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "clacroix", "validate_certs": true} 2020-04-10T06:59:27.581 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4', u'name': u'rishabh'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4NYxUHx8HMgbVIHadruN1kAJS5be0aZA9rvGtYDfwD5siKNfXFRyLG3thkjxtEi7DPYMxgbpFOXW9EQKh5sQ0ohsADZOi8FSIAfQlOQ+7HAEp7DjBtjxoqB31jJ1e293YtO+nqAhcOMyGfDx+71w5sS2EFqy7PscA9gD9aV2iMxIR/JD1LxXabiqj12jU5AOxr7ZQ1+gM9aroQuDpNxY4U8jBTxx5OtIYRkeI/aN2gQj1mXGMKGo0ItFn4lsXGLxGxMpa9K2UFw2dWQVTw9VNm79nixrCX+lC2nyaJPsRYZipXF1ID2NRk7oJ79dJ/uRBBXpunihlEC3aF+nOgEPH ridave@redhat.com", "key_options": null, "keyfile": "/home/rishabh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rishabh", "validate_certs": true} 2020-04-10T06:59:27.727 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006', u'name': u'epuertat'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAyiGmj+nOAD+il+FvXvZSpUOJQp6hKh6UPaAIQvQZkdeNRWN7wcDXMUlLj+TBFWa1OO+9xqFlnbgVMDR4qy5orUBYlMapUb1rutjbk64XeY58b6uguHwSoMd81QpCCIN4KOk3cldV8UiyX382baoZ9Je2hsOnnBGMsRtHuVHZ90Bj1rIITkwajq//KRs7kjb/qhjk0xkipDIcaSwKjzPlmmnYn69noODwDexpvLYlCF+LowSZEdrOUvVIigSFG7cLTDaU2tLhxSgbvWu8f0pwvjsyW7gOgM+VeRZ6uHC79F1qvgdgQSxflddIgrQpH0rzf6RNxQjR3SBkZ4V1BESa0w== epuertat@epuertat", "key_options": null, "keyfile": "/home/epuertat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "epuertat", "validate_certs": true} 2020-04-10T06:59:27.761 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2', u'name': u'skrah'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCeAhoTum3/VmRbxSyw5gwgcPF1EWFK5mEPX3PZyo3xPsUAJazALfge3aEuCrZBbsjOCoj3GfFhr/ziEF+OWvhBoPK5hQzP0V8tE9cWJwvHmD72vGE36o1hFO0ONE4DEuu5KRDqQJUgeyQWXsP26uHTwIcvMpy8egZOfyshqF+lQVXzsAex7Ninmidsf/sRNLhzT1kLrKT+ZMSbZ7Rqh46TMlmLD1LzJZNcb6PGVwlfQZmV6T7m0COu/skeSxMcy8hqMvHFNvvfFSqkfMw0KBGLDxMQr4L+e7sXwQDnEZHdzoh0iS3xDp/Onr1FSmPyEjz51e1sMHBsqklHWidSZVLt/+giB4o6Wdgzc0um/QxW2Ne2qjtDlViOgRE9Fzcw/cRu/Xsn8qtXTxYyxXswjkA8qX7WR22OYgjYChw9siUsKkuD2yAbPMVl+cZWDeBhQnPMbDMVaghS4SAVt5GK16jij4+r3IE9vGHBmGwJYJ/OrtcxBrUrDopwsGLcU+WHdv0FXuiw+YbfnzpthC8rVj4oeJ2IUOQ5JB0ub7NmbR4QWjc33JfD+AeCB75x6io07V2riRjMD9gLC5nwytJNG7Yytf6xI2i/CprW7HhsDs2ykOVcDSBMnL1oGW6FeHaDRW39r6BXSt1jjZQ2E6pDEBTDO41tPEL9EVvv9TK87GTylQ==", "key_options": null, "keyfile": "/home/skrah/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "skrah", "validate_certs": true} 2020-04-10T06:59:27.848 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461', u'name': u'tdehler'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRZyWiDO5JqWHJdUUBr8YReiKXxnL87juFAfkcmw2E9KmANVH2sn3K+OgUCQfBs1Wdg6AbcL2GyMm9alodUruzOm127/bH9OcGmu+8SNsMAYrAF9EzsKJa8Ba/I05rl5a87p1jbYG9Rpo7u3OvLx0LjgqydRYcmjwo/niifYNDTaN0K3v8s1mN4TorUZW/i3XBMBIBjlIiNGs/I7IZreR+1N4V8Nwrp2EWxDv7AwG3nvokL2RTHcsV0KgmAnciv8nXXjIf+uqOPuGA+F2qW/wz92bG8R0O4CjH+QZt4k6UoUIXFRW+LdOVAqmUrb4W0OJ8vsVeukW46SvPnYUTkZnF sonea@think", "key_options": null, "keyfile": "/home/tdehler/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tdehler", "validate_certs": true} 2020-04-10T06:59:27.885 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8', u'name': u'smanjara'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9/Io6M3q8yQqoPNPKAPJ6Fzq18DMfmNtxcD8GUWNo0xcMxd7XOlarvYc7INDQjkpzx6s+aqMme55pSUZuN5plW+nuFmRPqBaIxyeCnNjzCch0wbtEZe5dzT270FhzM1Tu4W0dw+WD6zrGjXayLNs2uxNp3UrWtMTBRzd0IfxJ1R2aJtiCE/HNxIZn+bhS/OmEPV1izz23MvKXD+nRaeJK6Aqrg1zbnpoNskz2fZ/AnmhnAMc4EHl/NUyW1mi4g81KcQMwE8poWmw1NpZje9gWFcISVClLvvhk/A6at/v90P7PisX+/ulTA/QYJYvGBjgbNtURYu/PWzYBOAEZQGfj shilpa@dhcp35-184.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDqRSG3i8QY7t9sp6I8vDZ7LVdk+DDrMhI1QmvU2z+YQhUGDBOvvO6YP0gUXPDyalyj0sJXiwLWPmkrOsbnR96YCUcIt7Ba6R0d0w85JxcJ1WyPcsg2UyOBwq4A+s7VAFZm6FEWzlaubixsNp04gKDqBjSjaqA1Kwx8/Tg39byay8Hn3VJeZ1SkRukvhykz2e6gl7ydfuJQVmEyO3pTrd5157ZqgvQRABT10wc3vR2m4eC65hU/v0ci7HMY9oVtGEdWVBapWMuiocrOPCRWF5zrMJwW4vzPOCHa/47hBEIWfOWJidUVzAawnM3nOe7PykvvDK/iP8B6JpxKTE74eLeh shilpa@fedora.lab.eng.blr.redhat.com", "key_options": null, "keyfile": "/home/smanjara/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smanjara", "validate_certs": true} 2020-04-10T06:59:28.005 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305', u'name': u'laura'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEBD7CxJJfbLW4jtuLnTwkxVbhCfh/JL3Qj8oeLEkny1tE7QCeXn1bP5bScWSgLH34Wh4TAuH+8/g/koFwcCy1YuoGBdBV4KV7O9MTnFngdqNY6LKzCipm27tnWZEa10hrF8dRFTmhracd6kmmVGHgvGQPVAVONmix+2IZ83wdj8XJSQHeE3PHFLO39tU+ez/T76QaiBLDVo4ctHVt+N9bBbHV6Cr8fyiUUU6YEKMlcedvxGpnMK8oeOSNuDeh+DldcM9KYzNGEOM34jGRIvJp57dxQDQCtPPW5hvBXg7yMyrv6TcJzDd2LFD6AC/Qw9yrEQPXXyBCDkAlCqDR0+rH", "key_options": null, "keyfile": "/home/laura/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "laura", "validate_certs": true} 2020-04-10T06:59:28.033 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41', u'name': u'bengland'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDpp2NWr5T6s+jzHiz/FfCKzN7ZBnUSzyE7ieuntzjeGotyupAAjVBoSpCJgNDybm+WkE4dShv5mttPJ2/q1UZCLRj3nN+y3NG333aLZuyugR0ht5qtvGVegg58PwQ8CMobR7ca7bl5UdEy0UIrM3d/qpDOuPkuE4MIJcrtZGICE8NAVnmHNOB9Sm4ovAYkewTuMAv40j2Dl4F9dD6bg+OJfBOJ9+Zj+MpowYlAILtEB9Dc20zB8GGUEtr9yfOVroblTYhvTxGYRaE7aVBe83DNwkWGbDc/qUNR6bpA04w/P+LclYVsVKRoHOs1EXHSwNMFromA8RsKS+rajwu5uRg5 ben@dhcp-185-182.bos.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC93wGgyyQmTxa7f+BZcYPUapTH2N1mnV1ikl9nzJzHhgrtJ6pv00LhtH1HqGhvoqnqZjBD5XRAKyjMRcMBT5MYcXXep5RqZ9sJ6Xz8rHkBjfl/OOQUeZ/ZyG/Ksr1jpGraN2xtrjQQzbqN1YvirCqfLk2exPvVeEk4CGnNybwKwCWob1uKV46ASrYeQK8pfHxrW7pTo+xdErHOvb7uumZm89/YisUJGicO1qSHBeuw1MxjhnD+fIcxHKWY5NNJ4qScW6a5AABk1JJsoHkh9IGCVaf/zJv8D3gil5SiM8LIBKaNa1fn6oObPl/Ky/wkiIo2gmUZawvAVA8iJnX9eUPl ben@bene-laptop", "key_options": null, "keyfile": "/home/bengland/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bengland", "validate_certs": true} 2020-04-10T06:59:28.162 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce', u'name': u'yaarit'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2k8xsrsLxEqJx3rV6ZZwLH0OjYCDB2uIYzcAyrKEOKlVWO1ICKxkw+gZ9tSiPFNLfq13io9BIrAKJwlrdLCJLmlgCQwpwUQqzmRlRZEm+WmD0dl8LPeQSidyJkPkJgYBEshFPjDHIj10Ei1pJS5svI2l7G1wMWxlt2kHL8Og/4zdFTRctxwqZcvzLmRQcs8lSSpqTkhfeKfmrpR+je8XFbCJJ9m/nJbgyCiZKaGN8nf36pCQDEva4rvaSrK0a6YxL0RWN8W3lqtBPKIehxffr3lYDEkD9EUW5vf6AUKC0ioh4jkPo9WCjaoFsOauMOMVQPHehEAMufEO4Kym6nfrf yaarit@localhost.localdomain", "key_options": null, "keyfile": "/home/yaarit/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yaarit", "validate_certs": true} 2020-04-10T06:59:28.174 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d', u'name': u'pnawracay'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC3YhI4am/T6s4rJRvbC/jAvrNcDhE/zaJiVDAjEuA1W/5Xt9ObX52eAnp+tOOtelXrX8ylYF/z5wGNQLjYrWepXhuqaEGxeVayCHzts6LMpmvyZ6sYnAw9HFO9rzrTawm70DuvWqY2Zt4KA6fM+lfXk3Qdpi5RCPDKk1jpXwSCyUtUKSWTDRtYtvvJTcO3H/xCJxQq/F9sU8yqgQcqFPfMNrV4a8gNj9KACqjt4IEIfuxKlKhlMA4yu1VTmGcNuhQS+geJzsEGuxZUQb0Io+q4o0b+40BQtazKgOdTcQbgNtqh3wZpjJsvB4AM+GNPjZY3mCsd+WPGg6dqfU08Voo2b7BfSwyCn14fQHnj1nSwRIea2juFY6AMfVJjstM5iIz7lOH2c3ydWWhzvSmMeoQ47zBXlg8JytWmDrXiyd21BSIpTL0Tn8P95wHYka8U9KRV0ZY7Jr2w/ThPdy2C+Cdir+P+OEL1sKHSnmEyQEeONa9P5yH1Rd1kM/FNESAIRiMLGspzXQeWoJhHswVaKoSBW/7f4b9VENxda8qAQi6ob8A71cAjbpRrkUmd5EBCSFMNs7lscus0/deUVvlpsrMjDAJa8S9gqhGHNU0YvF1WBIz3paLRAHDm9DWrHc5lreizJzX5ul4P/Z4ITjpO+9ZxKOcyi8wnJ8Og+QUYLpqOIw==", "key_options": null, "keyfile": "/home/pnawracay/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pnawracay", "validate_certs": true} 2020-04-10T06:59:28.321 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8', u'name': u'alfonsomthd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC709gIxs1RgiA95ddp2uymzgV84VHJVVMj71FyTlz6pa0gtlaxF+r16p8dnNWjhyJRUXMwBKFOwHx95bcdfiRA+BfC04UnI8VvBxoED5Nt9EbKD5h78ZoNEeQJKoG1dsM/kWWlTd9WCsCyzjRz32+YNWul0LOhrfloHP+LHnfjf/uOnzd8iRHgnpWr5Z9uB0MBu4y/L6AmHnR+8pMZQq6Bfno8ZLwNCzx9v+4MqOz3jY7FYv7SJIftQH0fJygIfCz3bKyUEWk9a/JJacETuJGBnHgNAXUr/FtM1IFiMl2SsTixRgnosEz4pX8EIe5LJyw/Qudro8BPEC8aIGy0XAkerGmTt5iALMnzmeeanZCFm4rF/YiFoHLMS/B0uv42MD64hB3knrV8Bcq+mRjwVOYgecO+xoiBsU6FZeXaBk5IsjmYTwataGv3ILEmNpss0WeqHxF9nd7oTLnw51cZfcaerOnJ5IZJZTXwLNVr7K176BunHggAhjvtvwdeKhphqt9f3ilLr6gzksEoAR4JBOYJJtd8sEqHwBtti+YbMGwH9Y+Z+SujlQBQIsom60xj5OzrwjsIi5JAtOhbc1Mq2fptSXRsR8ZBT8aUFqZ8R7ixEqq2F47OAznbhSQKidzwUNkWNI3WAKi38Oi4aya1k3iqtL04bGf8R/xD5Vq/2J+UlQ== vant alfonsomthd", "key_options": null, "keyfile": "/home/alfonsomthd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "alfonsomthd", "validate_certs": true} 2020-04-10T06:59:28.336 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3', u'name': u'rpavani1998'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUznqo3VDmcKRL64P7sbjuxXCW/0ljFTc5zZoZ9UvtZvU2jrRQ2X9GyxcrcQnAWlF5YZHqeTDU5o8ixn5fq1MSmaPRBNDKKVQOfR/vFQV73LiKx4BFsC1/EdaPie/XqltaLDTtfqhxADJlz1H3a3ZlkeVHMROTqo/TidL1YUYlRH4qHxWWBuFuVS+EYfTegSZdjwp+DuJA0iN6X99+DfH2DhiHENjEmlbKN13+H+EPzm0K3ya2w8D6ANodB3AxMn2tGs8mLtG/P4kCjoDnoUVFzKTyUpTtgOjis47uSY4dL8mTbIK9kZAiFxnT5sMClu6KRStDjRIrxbTBQJ8wvlat admin1@rajula", "key_options": null, "keyfile": "/home/rpavani1998/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rpavani1998", "validate_certs": true} 2020-04-10T06:59:28.452 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e', u'name': u'oliveiradan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCoL9+aMxLtUG3aRIuT4kbjc1CPuQ+wbS30ocuJ2YIExlKMSxVr+LYqbjcz0T4f8gAKG539dZQgD9BAsHM8yfOVEvei0KsQHn87pzgYGR3DtaMx5IEG5xNyARPu+VuEoe8qlL3cAPwTiaU78datD5Cz9GEClhL1Fhn/Z8o7BMbs5xlcUIQ/J+s9/sKMcqmmBJy5IHQIEUHCl4SFwsev5g5ADlnLNNxjpalExuBCEeoSma9F3b2YEJHvpA5JhCIMZpVfXbgh2UhUsyfAQN6KgOIQ/XlKBMQDtizf2vZ4b8hW4ELcvKSADfTsyyqEs/Ql8jEJu32GFKf2mYPbgTS6E7FZfQz77X+5OBhojFAstB8LSSrH80FrG1gCbVZLnW0jXcSehKnU0oNmUWOKxxaT1b00fo/iZ4P8O1uJRY17/uaVCPNKdGTZzIlzyC16wsFFBxBl0d7HfojQY6qfww7AQJhvDkV2sP0/MIJzwriWOzZEmfr8Osqery5vFiYcSvMbdMZDfMEYd3z7Va2RiTIeyXuEYp5hzf8eyJRP/f2FEw8/BBrm5JCxeSWUx7qwgPHrJoWAVxCIg/M3YS/vfVSFc8Hh88sgc2upq0OqYr0uq6EFzUZi4aoOkOVYn9u/uc/p1AR0KbiQOkERMUDWTyADxrHaL0nQmGy+XyiJQjhq5ckZ+w== doliveira@suse.com", "key_options": null, "keyfile": "/home/oliveiradan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "oliveiradan", "validate_certs": true} 2020-04-10T06:59:28.490 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4', u'name': u'rishabh'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4NYxUHx8HMgbVIHadruN1kAJS5be0aZA9rvGtYDfwD5siKNfXFRyLG3thkjxtEi7DPYMxgbpFOXW9EQKh5sQ0ohsADZOi8FSIAfQlOQ+7HAEp7DjBtjxoqB31jJ1e293YtO+nqAhcOMyGfDx+71w5sS2EFqy7PscA9gD9aV2iMxIR/JD1LxXabiqj12jU5AOxr7ZQ1+gM9aroQuDpNxY4U8jBTxx5OtIYRkeI/aN2gQj1mXGMKGo0ItFn4lsXGLxGxMpa9K2UFw2dWQVTw9VNm79nixrCX+lC2nyaJPsRYZipXF1ID2NRk7oJ79dJ/uRBBXpunihlEC3aF+nOgEPH ridave@redhat.com", "key_options": null, "keyfile": "/home/rishabh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rishabh", "validate_certs": true} 2020-04-10T06:59:28.584 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba', u'name': u'swagner'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDh9bzZJulXGES+l9Xh6Aq15RZ8uQCGuDNhlNQtDblE1ISKJ/DqGYXp6wUW54+oVNA7eZiXz+fi3mq5pPEtZZOfd3ixEDzDJ4E3cVXoDZCqeWEmea6KvybeY10YxvB56TEI8U2KKAd56PRl20klXpLCjzXNqG7n0aXFcpCMbXDu369VX6lOk24K/7++7Wc5SttcvVu19sT2kqzsB/S1Y5cxiE6RM6wVtqBoksp/kRCIA16ruNwx3GUabDfbEoUGXlkkUP7+TZgAbHtBsYy6mCQhwi0S2+WG+HhHDUPjHhV+MdN9ffibCtOEGo52itVLVky09VeBocuA6H22JDGXPBgjcgf2NsZIqcKqGHhUkXmH92fhRSFOBLKHstrBq8jWRP/mNrgj8cQksDsakQYQbDg5dyabp+M0/iL2Q3YVq7erZI8aZMA7ZF3WgoQNYZg5E7oejM8URIlFP3x1ne2ClRC9a74phSCxeU/NVamGN3G3dImzEXGOSNyRggHJ4jGrIGc7tLPCzmI5OkomcB5OxReqf0r1TNXuUAqw8M4EoWt+0xoAmH5zVlUHf+psUCJIEV/4pbgtoJiSNq+LVY4jyDEFbvTAL7MqyXorMV7Tqlj+/3d7RXfhW9lR/SHb1Z3jcHvz4ZzYMzKWeJEjz+Y0NwIdFDhcPmUOYtEDmuRRrgYiSQ== sebstian.wagner@it-novum.com", "key_options": null, "keyfile": "/home/swagner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "swagner", "validate_certs": true} 2020-04-10T06:59:28.649 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2', u'name': u'skrah'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCeAhoTum3/VmRbxSyw5gwgcPF1EWFK5mEPX3PZyo3xPsUAJazALfge3aEuCrZBbsjOCoj3GfFhr/ziEF+OWvhBoPK5hQzP0V8tE9cWJwvHmD72vGE36o1hFO0ONE4DEuu5KRDqQJUgeyQWXsP26uHTwIcvMpy8egZOfyshqF+lQVXzsAex7Ninmidsf/sRNLhzT1kLrKT+ZMSbZ7Rqh46TMlmLD1LzJZNcb6PGVwlfQZmV6T7m0COu/skeSxMcy8hqMvHFNvvfFSqkfMw0KBGLDxMQr4L+e7sXwQDnEZHdzoh0iS3xDp/Onr1FSmPyEjz51e1sMHBsqklHWidSZVLt/+giB4o6Wdgzc0um/QxW2Ne2qjtDlViOgRE9Fzcw/cRu/Xsn8qtXTxYyxXswjkA8qX7WR22OYgjYChw9siUsKkuD2yAbPMVl+cZWDeBhQnPMbDMVaghS4SAVt5GK16jij4+r3IE9vGHBmGwJYJ/OrtcxBrUrDopwsGLcU+WHdv0FXuiw+YbfnzpthC8rVj4oeJ2IUOQ5JB0ub7NmbR4QWjc33JfD+AeCB75x6io07V2riRjMD9gLC5nwytJNG7Yytf6xI2i/CprW7HhsDs2ykOVcDSBMnL1oGW6FeHaDRW39r6BXSt1jjZQ2E6pDEBTDO41tPEL9EVvv9TK87GTylQ==", "key_options": null, "keyfile": "/home/skrah/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "skrah", "validate_certs": true} 2020-04-10T06:59:28.723 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e', u'name': u'yuvalif'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDzYzqIDimPjMOUOUz1A122eV52fl3mgGgNBRYRlOWl/lOxM5oOat5FPkJlnXTVyQFl0MlnYgIv9fTnjHVTM0wFmFUItlUl151ZmK5XExiGpGVsqG3UWNyKJxZpk9Uh2a0T/5CKzzwXnv2fJMNxzfqgopYDrgq4TFEZcdvgWJ39dcRTXST/rGJQ8wOq9z7UNzKU5VOGxvgi6f/nGevGIHrSaTcY3Uu8BW8kXUrf9vfl+ZSxVjuNHXF/VmrJH5Ne8mVu7DzPMrl8ZZjxsfJKT/eQqF9kjqht5586OSPKIwg/LEOyYj7Wyyf3QbDAIp5bCygQfmBBl+kIxphJYq3EwUZT ylifshit@dhcp-4-102.tlv.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFgwQYD9amVSYq6JzjHyGmKBJATQCM9xsf6QnF3yb0Lvry2nzY9F61evVXQ95TmF/AqasRx5p2WansIvegG5ag5mytvRwFXtHzPs4o/J+XjA9o4+A5l6j9cOFLIDD/TwrwlUjgMa2PhqUztSpI1op906391RLrJAyflfDghJxeVl6dJab6TEsslV0W+yaCS3BBxMXXrCbxCB5Ms3KCRKDz0DqbhFc+Nw05L1O1pC98HBFRIp1SaXlMcT3++CRQ3wugpOMkDMNLbycl+vdMBFY3UZ28rRowS3Ta7V5EARZVVH7KxU5cL+8Hr/8DE1/ITOmrluaEZ1KwYgoxZbZ3w7Ptw0FK6aiCZ49YtqKT1WakoVfO8m7pCdxkqUOA91loMQtxvCwZ/Vfoxaut16gklmPz8N11xvo1H5XB8/ZrvK5yKdl3JDl+wwtfE/fwo4E/fte+rPRisoWBRsmsx8EGxs75sXnb6n3VjZ65vYEKsNMLg+5EAaevWLLeqF0a42UWGJrHdb4VKX/OAplHrOk7kA/VwArow8f9n+iaRp2hgDU9OhQiFVRhgqwahlmCFFXkcP4/4WlxMCuZrsaNMZgMjVhfsp7cskKOezSfZZey2MB2fREMS1wg0j40CdYRn4jTNEBr/LVszsoM7t7AwXUpnqXeYIe+gePx1PJb5XSzSPZ9Pw== yuvalif@yahoo.com", "key_options": null, "keyfile": "/home/yuvalif/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuvalif", "validate_certs": true} 2020-04-10T06:59:28.790 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8', u'name': u'smanjara'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9/Io6M3q8yQqoPNPKAPJ6Fzq18DMfmNtxcD8GUWNo0xcMxd7XOlarvYc7INDQjkpzx6s+aqMme55pSUZuN5plW+nuFmRPqBaIxyeCnNjzCch0wbtEZe5dzT270FhzM1Tu4W0dw+WD6zrGjXayLNs2uxNp3UrWtMTBRzd0IfxJ1R2aJtiCE/HNxIZn+bhS/OmEPV1izz23MvKXD+nRaeJK6Aqrg1zbnpoNskz2fZ/AnmhnAMc4EHl/NUyW1mi4g81KcQMwE8poWmw1NpZje9gWFcISVClLvvhk/A6at/v90P7PisX+/ulTA/QYJYvGBjgbNtURYu/PWzYBOAEZQGfj shilpa@dhcp35-184.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDqRSG3i8QY7t9sp6I8vDZ7LVdk+DDrMhI1QmvU2z+YQhUGDBOvvO6YP0gUXPDyalyj0sJXiwLWPmkrOsbnR96YCUcIt7Ba6R0d0w85JxcJ1WyPcsg2UyOBwq4A+s7VAFZm6FEWzlaubixsNp04gKDqBjSjaqA1Kwx8/Tg39byay8Hn3VJeZ1SkRukvhykz2e6gl7ydfuJQVmEyO3pTrd5157ZqgvQRABT10wc3vR2m4eC65hU/v0ci7HMY9oVtGEdWVBapWMuiocrOPCRWF5zrMJwW4vzPOCHa/47hBEIWfOWJidUVzAawnM3nOe7PykvvDK/iP8B6JpxKTE74eLeh shilpa@fedora.lab.eng.blr.redhat.com", "key_options": null, "keyfile": "/home/smanjara/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smanjara", "validate_certs": true} 2020-04-10T06:59:28.791 INFO:teuthology.task.ansible.out: 2020-04-10T06:59:28.853 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852', u'name': u'kkeithle'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXa9uTf3q+7tGnHzSxv0WbgpRKrBIYjAZ5x3V74vQUSD1aAbmyvOqgPnts7XttLYImjjRqiisyC03L3KgR4b3SOeyL3KFxrk3KkHAJ/AJH+Y/UafyDhHNl3gRNg0B9rjsa7h2dSlcrFKnAX0pDY1HapuNPrzYWrXsEbo7C6wteFh7PyPmTkHu0NzIIPC4FcLl8SGaww/HeBh8v91IymgFSgXG0jNC063FEExWdti54yNLGh3gpW0FytsiRN9eEOKUOgcveT1tejz9ORv8evO4Y9SCQ/lsyOKQKNcG5C7J8nsbwSjqQ2cf6i2aYFZ0Oq1UXuhJ8APJCoiKgLJy7AiYf kkeithle@dhcp-100-19-89.bos.redhat.com", "key_options": null, "keyfile": "/home/kkeithle/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kkeithle", "validate_certs": true} 2020-04-10T06:59:28.930 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41', u'name': u'bengland'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDpp2NWr5T6s+jzHiz/FfCKzN7ZBnUSzyE7ieuntzjeGotyupAAjVBoSpCJgNDybm+WkE4dShv5mttPJ2/q1UZCLRj3nN+y3NG333aLZuyugR0ht5qtvGVegg58PwQ8CMobR7ca7bl5UdEy0UIrM3d/qpDOuPkuE4MIJcrtZGICE8NAVnmHNOB9Sm4ovAYkewTuMAv40j2Dl4F9dD6bg+OJfBOJ9+Zj+MpowYlAILtEB9Dc20zB8GGUEtr9yfOVroblTYhvTxGYRaE7aVBe83DNwkWGbDc/qUNR6bpA04w/P+LclYVsVKRoHOs1EXHSwNMFromA8RsKS+rajwu5uRg5 ben@dhcp-185-182.bos.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC93wGgyyQmTxa7f+BZcYPUapTH2N1mnV1ikl9nzJzHhgrtJ6pv00LhtH1HqGhvoqnqZjBD5XRAKyjMRcMBT5MYcXXep5RqZ9sJ6Xz8rHkBjfl/OOQUeZ/ZyG/Ksr1jpGraN2xtrjQQzbqN1YvirCqfLk2exPvVeEk4CGnNybwKwCWob1uKV46ASrYeQK8pfHxrW7pTo+xdErHOvb7uumZm89/YisUJGicO1qSHBeuw1MxjhnD+fIcxHKWY5NNJ4qScW6a5AABk1JJsoHkh9IGCVaf/zJv8D3gil5SiM8LIBKaNa1fn6oObPl/Ky/wkiIo2gmUZawvAVA8iJnX9eUPl ben@bene-laptop", "key_options": null, "keyfile": "/home/bengland/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bengland", "validate_certs": true} 2020-04-10T06:59:29.007 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130', u'name': u'emmericp'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBYa5uEOlFv1s9K9D1SbcaApLwX3LBwASSnRXDHI7MtU paul.emmerich@croit.io", "key_options": null, "keyfile": "/home/emmericp/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "emmericp", "validate_certs": true} 2020-04-10T06:59:29.107 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d', u'name': u'pnawracay'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC3YhI4am/T6s4rJRvbC/jAvrNcDhE/zaJiVDAjEuA1W/5Xt9ObX52eAnp+tOOtelXrX8ylYF/z5wGNQLjYrWepXhuqaEGxeVayCHzts6LMpmvyZ6sYnAw9HFO9rzrTawm70DuvWqY2Zt4KA6fM+lfXk3Qdpi5RCPDKk1jpXwSCyUtUKSWTDRtYtvvJTcO3H/xCJxQq/F9sU8yqgQcqFPfMNrV4a8gNj9KACqjt4IEIfuxKlKhlMA4yu1VTmGcNuhQS+geJzsEGuxZUQb0Io+q4o0b+40BQtazKgOdTcQbgNtqh3wZpjJsvB4AM+GNPjZY3mCsd+WPGg6dqfU08Voo2b7BfSwyCn14fQHnj1nSwRIea2juFY6AMfVJjstM5iIz7lOH2c3ydWWhzvSmMeoQ47zBXlg8JytWmDrXiyd21BSIpTL0Tn8P95wHYka8U9KRV0ZY7Jr2w/ThPdy2C+Cdir+P+OEL1sKHSnmEyQEeONa9P5yH1Rd1kM/FNESAIRiMLGspzXQeWoJhHswVaKoSBW/7f4b9VENxda8qAQi6ob8A71cAjbpRrkUmd5EBCSFMNs7lscus0/deUVvlpsrMjDAJa8S9gqhGHNU0YvF1WBIz3paLRAHDm9DWrHc5lreizJzX5ul4P/Z4ITjpO+9ZxKOcyi8wnJ8Og+QUYLpqOIw==", "key_options": null, "keyfile": "/home/pnawracay/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pnawracay", "validate_certs": true} 2020-04-10T06:59:29.139 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0', u'name': u'mchangir'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCj28V9sMybkAzhkiaNiDDr8TfNVyr5DNdbOqszdGEVLy1Rxh/UYnxUclsR9n8lt9VnCft3M4NDTnVwlxOHE8+0oolJE9HP+DQomsgj3abP04EGJyOA0X8E7eIFQNLRbMFCD0yX/SNNlYPsS5fPTFhIa/gbBPmDBE07kULts66Tvc0Yo/2yIfwV1DV34bcWWVGi01XybIHBl9qbvlfBKAUrFEcv3ZdpHNhuZw6IkZY8xoYxo8iF+toLYeGX+gFxqCZUUOO+NBVmb0hVL1LI6bU66IrMOQDkFurqpvoraBTJqwZgXvMPceFUkcVIH4Nm7cK1X2WyMtnSw2XBpLMT8C4V mchangir@indraprastha", "key_options": null, "keyfile": "/home/mchangir/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mchangir", "validate_certs": true} 2020-04-10T06:59:29.256 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8', u'name': u'alfonsomthd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC709gIxs1RgiA95ddp2uymzgV84VHJVVMj71FyTlz6pa0gtlaxF+r16p8dnNWjhyJRUXMwBKFOwHx95bcdfiRA+BfC04UnI8VvBxoED5Nt9EbKD5h78ZoNEeQJKoG1dsM/kWWlTd9WCsCyzjRz32+YNWul0LOhrfloHP+LHnfjf/uOnzd8iRHgnpWr5Z9uB0MBu4y/L6AmHnR+8pMZQq6Bfno8ZLwNCzx9v+4MqOz3jY7FYv7SJIftQH0fJygIfCz3bKyUEWk9a/JJacETuJGBnHgNAXUr/FtM1IFiMl2SsTixRgnosEz4pX8EIe5LJyw/Qudro8BPEC8aIGy0XAkerGmTt5iALMnzmeeanZCFm4rF/YiFoHLMS/B0uv42MD64hB3knrV8Bcq+mRjwVOYgecO+xoiBsU6FZeXaBk5IsjmYTwataGv3ILEmNpss0WeqHxF9nd7oTLnw51cZfcaerOnJ5IZJZTXwLNVr7K176BunHggAhjvtvwdeKhphqt9f3ilLr6gzksEoAR4JBOYJJtd8sEqHwBtti+YbMGwH9Y+Z+SujlQBQIsom60xj5OzrwjsIi5JAtOhbc1Mq2fptSXRsR8ZBT8aUFqZ8R7ixEqq2F47OAznbhSQKidzwUNkWNI3WAKi38Oi4aya1k3iqtL04bGf8R/xD5Vq/2J+UlQ== vant alfonsomthd", "key_options": null, "keyfile": "/home/alfonsomthd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "alfonsomthd", "validate_certs": true} 2020-04-10T06:59:29.268 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286', u'name': u'sidharthanup'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC1OVCo0wMITAp8PNUqsTRPFz0SuBL0r1RzKFJUoxQm7gsP9qttZUQRlfiuZ7YRzkuGLOEBngPuIcV995o0lZ0+LVHA8jxWEJamb6sM9YgnonbGKDHfl4lT8LJd4YtjcWnMGvUUdPiPgIkfrpxXtx+RzJ7eCTmgRjAJziZrRh3AqXuKty+s9MDnmK8EQ2smpLdkMtZG5W3U2PiVaGgq2FIIOAoqr58xGqFgZ4tIpvWFpNxVsDZXIg2NdC+ouuKQvuJRWq9eERRHwq26CR+kDFNZeYSyhYKxNBqauLCsAoXNFCJzrdX5Oi9Ox4lJTsEtYX+SMziVipQS54F7MnRAUccukQ86IbXz8C5Ok2EzNo6smwRZ6UPKRvebWjIvfbeR/iwPRnrWcNgjGcaTR6yhlRJh8pBkW84nyY/bJa4WXsoNLgutdsYkjl18KKhbBldjdf0ovfL7FVgqYfAo/wpQYJJeMebP5QAczxvZIegjHRrAvbs/nbQTmuAkes3WL1hhBjSTwyIbWCzAuBJk8MNTmeZkVBvub7XoIyIySppfoJC/1a+ZsxGkzAlgrIxXPYrrBzTtXd/9YRiIgCYx3UtfdbPkE6hT6GblH6ST6QESfWyyvfGxwlOmu+F+icRyVVJbgM9T/PD2W6I04u2/1mcfoHwaQGdwvbbBkbjWzm+zwNSisQ== sanupkri@redhat.com", "key_options": null, "keyfile": "/home/sidharthanup/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sidharthanup", "validate_certs": true} 2020-04-10T06:59:29.421 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e', u'name': u'oliveiradan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCoL9+aMxLtUG3aRIuT4kbjc1CPuQ+wbS30ocuJ2YIExlKMSxVr+LYqbjcz0T4f8gAKG539dZQgD9BAsHM8yfOVEvei0KsQHn87pzgYGR3DtaMx5IEG5xNyARPu+VuEoe8qlL3cAPwTiaU78datD5Cz9GEClhL1Fhn/Z8o7BMbs5xlcUIQ/J+s9/sKMcqmmBJy5IHQIEUHCl4SFwsev5g5ADlnLNNxjpalExuBCEeoSma9F3b2YEJHvpA5JhCIMZpVfXbgh2UhUsyfAQN6KgOIQ/XlKBMQDtizf2vZ4b8hW4ELcvKSADfTsyyqEs/Ql8jEJu32GFKf2mYPbgTS6E7FZfQz77X+5OBhojFAstB8LSSrH80FrG1gCbVZLnW0jXcSehKnU0oNmUWOKxxaT1b00fo/iZ4P8O1uJRY17/uaVCPNKdGTZzIlzyC16wsFFBxBl0d7HfojQY6qfww7AQJhvDkV2sP0/MIJzwriWOzZEmfr8Osqery5vFiYcSvMbdMZDfMEYd3z7Va2RiTIeyXuEYp5hzf8eyJRP/f2FEw8/BBrm5JCxeSWUx7qwgPHrJoWAVxCIg/M3YS/vfVSFc8Hh88sgc2upq0OqYr0uq6EFzUZi4aoOkOVYn9u/uc/p1AR0KbiQOkERMUDWTyADxrHaL0nQmGy+XyiJQjhq5ckZ+w== doliveira@suse.com", "key_options": null, "keyfile": "/home/oliveiradan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "oliveiradan", "validate_certs": true} ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31', u'name': u'varsha'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCkeYXWjU7wVdODC32U40QRFgFH7ZEuEsOeWWaKx3Y5EBuAcwkTHiF9Bq/2mrDu+c9W46S8L38Ogc9GZ0DcCXtN/Z4z8iKOIIi5U1GzvwdbKuiAPCd8bdYP0vo0MOIRgylJo5hcnB5B 2020-04-10T06:59:29.422 INFO:teuthology.task.ansible.out:CwCXRN1hmci1ES5UbSeQg0rydMiyYXHZ64AT47+qVUpn/mVP6PVw00pPqt16l9YxhbPFG74Y/WnY4FyrqXGQJfv0tcP7+UR7GQdFiI4QQV3ZUSRHRLWGcmoiiOmGimLhtzEmP4Jzy4gF4wg9q8jyX54NjfxDRZs+YuaoqqWKz/HSz5ic7W2RGKHrRby2N9wNu19nmC1NzIn3 varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCW58OUR3pniaYgvr9xAExeNm4eMxgaJfFbEfEYPztTWv/xxqqihBYCeFltfgWbu7ERTy+hVNo1PJWyyczjl5Ivc3FdRmhykosygs7X74PPh02c7Em4MsSTr7SSPNN20rkJbWqiYMMOBbqbiZ7JXiSUNVilLHgZqZBEQc5Rf5I1PyBLHHJ2ZruPKnDPHCRLdwJ+MdXy2haKmGrQLhtzIKaDzh8VkfojL454k0aHCIh34Z0oBdP9757mNuYJUVEkKZF5t2oML84lWGMgwZxUWN2yO9ALTti4tNH1fXA6NipS9ztE+3hEJe1fWXEoUQDX9JNu4qEHTeCz0secQMkyZVDZSQk8yDUQxQ76yiett37sQB1WQR/3fclcahmqQ4dUS7M3Onyz6k6zia896PBK6G48TBUbFa1rw5Hx7WfNZluDMkv4YxkQvfz7XVzZx2/lhs45Kjh1UR9n7LaW/NID98vokcvASw1N6Bk/6pOd/45iPyEJemwovnQLs6ecwlMIikM= varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDcQ8xMjwVf+KiFIQARcbFT+wg4NHysBRrFxk3Bx5X2k2hGKcEYR2tY/ApHZYfJ6me3LUvJEXg2c/6xa6ibVTUp/90l4y0cF6RmF5uB45J0xuBYoFJd8xJkVrhDIlZix4aFfb7TrmJEqjleBrkTwam7Q9aE3lss16nVOxU4hiViYiCANJ45wobK6tz7y5btfzrHE/iYpwniDvzLwK7EpkEK336VVd6f9wV0vG+ErEFxnNVxheXLLm05LuvdVQvFqJcTFrc7/GZ54yKg5IgwVwXqSAyOXIhgduUqk+oN1ciA7jnSqdiOKQPS/VmYkbhw2YP85l8wGVreIYUmOG19QEyVLB82oB9KS1LezM2KCH61XW22+85dHf4PvuL9DULhGt6wBCvpO10OHxw0JvHXbEgBPhiyd9j2yJmcQozbRPth/sMUvLKNXOGabZhhkPBYL7xZICHc+UwxrXs0h/j9jOjLyHzBAWMomF2lzgbvtzjcbxJ1a5oclJ6IW8yHJllkOh0= varsha@localhost.localdomain", "key_options": null, "keyfile": "/home/varsha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "varsha", "validate_certs": true} 2020-04-10T06:59:29.557 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n', u'name': u'sjust'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6RDgqwA0GrvE6KqRaGCgYnFhnxdi9ehjIWvfaMa1oq0CqYxUAee2bNbp/X2Bd18Vqk8iH9fpE0+qrLgAH/8BpD+MtgGikCFMORvazeUtOCF/OCDO0AMg1jvl8aTdvrQU8ZTDPXa65Avpkc9vwugcoplnwrW7lKx++XqCWsSlfHgEdyMV3wrt/ZDFwI2AS2bZBpWM6ViG1KSjXjgTi/T3hSZPY+buYvWjzoEjQO0946iI3iiKKCGohEpCUnG02oJXcS0vvgl8dWE71451lbainEKJ4hbDhfDmy57s6VXfVUANsczwiPWiSC/ZmMlDaCCBeKHO0Gz0xXaDXd9j4SA7P sam@Pondermatic\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYylAUS12hF6XndDQ3CkrUddKmXNSd8pOlHwO+pWaDTJPKxwtPL6sM3aOtaAPLrGkv1zD2k3iZaxmDwfPYGaXOyb0KS5ZCddtOyNXj5JLrjxh2niqFPP8SIkdHLOP+0Hv1/tB0Jz75RV+tNIT1SJTm7OZ8wZNy0qYycITSooJ14iEgyZ7rOa3EEzbtIVpHS4+YqIuIJdV2oV5egCrJyRv4sPUrS8ERXD4QZSOlHE+W4Tf+WVrEfsHOdDLRlAXJUzhuqgWtVouiYcW5ex294z28eA+12lKaPQpE0cft/NJ6FLs41yqh4ZN588bl8Ada+yTXEaHWioQEEJMlca2jIgrP sjust@rex004.front.sepia.ceph.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC08RrDQ2M7Zjcpx7RL9e+tjv48kBgDNyTenX1QdxW22JA2JR3D3ThA/ej9HtiPWcIVz16oZ4tzrCEVwaCMfb 2020-04-10T06:59:29.558 INFO:teuthology.task.ansible.out:smXq2z08ZXYqwiBuoXJVshU7lIKVqAtadT4F6Q9eQvxEo4zWdBib1W0/cqNLlYzU/DuwlUqYnWBDLGxc4fD5Kxrz366oWUDB/ULCpl77TMqfTKlStCgGXot0zLqiscqzq1W7XsuA7A5vQw/JTq7MvIME+SkUuGWX0WKwTMIRZKNWBGmlf51bNru3GmmAnKBUFsJv9H4G0FPqGY/aH02rr5dAgTTmfKt+XbWRfMTMpjJqVRGX03+1mQNoYQ3kNDz0wl sjust@office\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA5UMPySf/h9r2/OryRXrO9Qq9k5J9Dn9nvJYyaYCE82H/qkBJGboAJgyIXfbsGYO5uPM25d3fGRH4magzAQm2hGWNbbt1YWn9HaIyKCCXo9JX600/dv3OZbHyGGmgXPa4Anl16j1c2vxRRt2+ArjlyyhP7cAcRkilUGdagK9u6ALrwnz9hhXYV+BmXrxMm7zZCmdWHcWz+YxoA65es4+w+aLvVZWmUCZLF8uZrcpPSw+oK3SfZE8JPRc5vWulXbWEEfIoHb9AszPIpSyzbfPWXelmc3XlG0KD66t7w+AnNlQPt3JVePRCfFHPYeTj9B0k8sxPyUqTQIl5pu6GppRMOQ== sam@Deepthought", "key_options": null, "keyfile": "/home/sjust/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sjust", "validate_certs": true} 2020-04-10T06:59:29.569 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba', u'name': u'swagner'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDh9bzZJulXGES+l9Xh6Aq15RZ8uQCGuDNhlNQtDblE1ISKJ/DqGYXp6wUW54+oVNA7eZiXz+fi3mq5pPEtZZOfd3ixEDzDJ4E3cVXoDZCqeWEmea6KvybeY10YxvB56TEI8U2KKAd56PRl20klXpLCjzXNqG7n0aXFcpCMbXDu369VX6lOk24K/7++7Wc5SttcvVu19sT2kqzsB/S1Y5cxiE6RM6wVtqBoksp/kRCIA16ruNwx3GUabDfbEoUGXlkkUP7+TZgAbHtBsYy6mCQhwi0S2+WG+HhHDUPjHhV+MdN9ffibCtOEGo52itVLVky09VeBocuA6H22JDGXPBgjcgf2NsZIqcKqGHhUkXmH92fhRSFOBLKHstrBq8jWRP/mNrgj8cQksDsakQYQbDg5dyabp+M0/iL2Q3YVq7erZI8aZMA7ZF3WgoQNYZg5E7oejM8URIlFP3x1ne2ClRC9a74phSCxeU/NVamGN3G3dImzEXGOSNyRggHJ4jGrIGc7tLPCzmI5OkomcB5OxReqf0r1TNXuUAqw8M4EoWt+0xoAmH5zVlUHf+psUCJIEV/4pbgtoJiSNq+LVY4jyDEFbvTAL7MqyXorMV7Tqlj+/3d7RXfhW9lR/SHb1Z3jcHvz4ZzYMzKWeJEjz+Y0NwIdFDhcPmUOYtEDmuRRrgYiSQ== sebstian.wagner@it-novum.com", "key_options": null, "keyfile": "/home/swagner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "swagner", "validate_certs": true} 2020-04-10T06:59:29.698 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'deepika@x1carbon BgvhtHMrCcelT98ekPDMoA 6fa3cf08186b93b966b68841f3507a0809d348c5097b39b6140832cac2afd511', u'name': u'ideepika'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon BgvhtHMrCcelT98ekPDMoA 6fa3cf08186b93b966b68841f3507a0809d348c5097b39b6140832cac2afd511"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDj6DxTZNodVnwJZrnSZ/jv9Cw4uYLb8MGpccmOvtwl93sVBvqq597AYnBqYiV4VBbKU3GVFbT41zkplia1iCfXoddttCOyWmEKviBXRnAsd5f80dxL0pfzrBJvCaH4NAfYgVw9JFWsvT981b+nB6vum4Vz8Dw0PM5GPnVUjsZNFkpmDPUBL4yHPmMCGP2WtUurJVD3Gm9t72XlG6Dl/vB9Qw/i4IY1SoCCbuhhNgE7IN8JXeGmVJbRwDeUlP+Q8Ti7iedV7YB/W1opt6eT5EPzX8NcxTc0c7rrDgsUWjFQYRSup0GLK9ofdHs0E2RbPPO5OSc5Z2+B1IEV7fZeyJwAyHwUgwx7sSGy7aROdvinaj/hOBglcgISyQ2CY7LTV8NGWJquo7yRCJXhZVIHfgqfBcrgWdNbr+vNBXnWe5PCtLhV8ryHPSJ/7Ru+7voQxgvwux5SdZwxXTbWDKni1f6/8bNrhRx6bmLK9xuYAgt2FhlIivI7L5kQF4etcBiaf68= deepika@x1cabon", "key_options": null, "keyfile": "/home/ideepika/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ideepika", "validate_certs": true} 2020-04-10T06:59:29.730 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e', u'name': u'yuvalif'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDzYzqIDimPjMOUOUz1A122eV52fl3mgGgNBRYRlOWl/lOxM5oOat5FPkJlnXTVyQFl0MlnYgIv9fTnjHVTM0wFmFUItlUl151ZmK5XExiGpGVsqG3UWNyKJxZpk9Uh2a0T/5CKzzwXnv2fJMNxzfqgopYDrgq4TFEZcdvgWJ39dcRTXST/rGJQ8wOq9z7UNzKU5VOGxvgi6f/nGevGIHrSaTcY3Uu8BW8kXUrf9vfl+ZSxVjuNHXF/VmrJH5Ne8mVu7DzPMrl8ZZjxsfJKT/eQqF9kjqht5586OSPKIwg/LEOyYj7Wyyf3QbDAIp5bCygQfmBBl+kIxphJYq3EwUZT ylifshit@dhcp-4-102.tlv.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFgwQYD9amVSYq6JzjHyGmKBJATQCM9xsf6QnF3yb0Lvry2nzY9F61evVXQ95TmF/AqasRx5p2WansIvegG5ag5mytvRwFXtHzPs4o/J+XjA9o4+A5l6j9cOFLIDD/TwrwlUjgMa2PhqUztSpI1op906391RLrJAyflfDghJxeVl6dJab6TEsslV0W+yaCS3BBxMXXrCbxCB5Ms3KCRKDz0DqbhFc+Nw05L1O1pC98HBFRIp1SaXlMcT3++CRQ3wugpOMkDMNLbycl+vdMBFY3UZ28rRowS3Ta7V5EARZVVH7KxU5cL+8Hr/8DE1/ITOmrluaEZ1KwYgoxZbZ3w7Ptw0FK6aiCZ49YtqKT1WakoVfO8m7pCdxkqUOA91loMQtxvCwZ/Vfoxaut16gklmPz8N11xvo1H5XB8/ZrvK5yKdl3JDl+wwtfE/fwo4E/fte+rPRisoWBRsmsx8EGxs75sXnb6n3VjZ65vYEKsNMLg+5EAaevWLLeqF0a42UWGJrHdb4VKX/OAplHrOk7kA/VwArow8f9n+iaRp2hgDU9OhQiFVRhgqwahlmCFFXkcP4/4WlxMCuZrsaNMZgMjVhfsp7cskKOezSfZZey2MB2fREMS1wg0j40CdYRn4jTNEBr/LVszsoM7t7AwXUpnqXeYIe+gePx1PJb5XSzSPZ9Pw== yuvalif@yahoo.com", "key_options": null, "keyfile": "/home/yuvalif/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuvalif", "validate_certs": true} 2020-04-10T06:59:29.828 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46', u'name': u'gsalomon'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDC0h+TpaUkm2V6Hbf7aPCTys/nsm51CmY9s+bdwEXAiSAWpq/7Xi2LlN0mZgsrAYBHbr5L+Pz8x0YdigvWysUK6U9XRSPM28cbWmh8F25re9HEk8ARESkMZKTv/hKFs1v7kTHbT9hNNynbQGQOqEaazjr0shcli+Byvop27riX3QDDSsd5xfCjjyDz9fiA9YESlF0kbXyFUKPsoaBhRv9FUhtH6DxlygxtxTsKyKgeoiKw2sCfeUMckcJKnGCMtkXW4U8h7NL/fPZFyBaMDRi/TtoBKaVGLaWqF7RuukfjoMCwhuIDmuTG3WhanH4HJrfNenPZkpZUBhzWwZcleao8SD3mXXfZ9gcZb8QB2ymEtTqrAEyDoNWO4q8ljNP3WJXIgF0h0Sjmx/CV/i0ArEYlBMwBD7DFVnPJqCqSyypuQdDGsDNQZxaRCiYg7nIslFOpDr+0PLp78m+CHyYbSFLTvVVJoJ1k0YhMnDPHM29nt7fMGx6Qrigwnuk0yKg1bfgJ/t0TOqEw0bqGKIvHnyoP/n0ObazEsiPl5pRr2ehs/Rqf35Iwhc1i02Hhz/3Sqwc1eAgqm4/OVghxqUJGpRocua5Gf/vO32wwoyGxh2sl7y6K2pNDA3glhKYbZeIh1fjnaTjikgleqIQkCBxbTeAE7n5+T6TcFtnII6RSxYv3tQ== gal.salomon@gmail.com", "key_options": null, "keyfile": "/home/gsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gsalomon", "validate_certs": true} 2020-04-10T06:59:29.861 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852', u'name': u'kkeithle'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXa9uTf3q+7tGnHzSxv0WbgpRKrBIYjAZ5x3V74vQUSD1aAbmyvOqgPnts7XttLYImjjRqiisyC03L3KgR4b3SOeyL3KFxrk3KkHAJ/AJH+Y/UafyDhHNl3gRNg0B9rjsa7h2dSlcrFKnAX0pDY1HapuNPrzYWrXsEbo7C6wteFh7PyPmTkHu0NzIIPC4FcLl8SGaww/HeBh8v91IymgFSgXG0jNC063FEExWdti54yNLGh3gpW0FytsiRN9eEOKUOgcveT1tejz9ORv8evO4Y9SCQ/lsyOKQKNcG5C7J8nsbwSjqQ2cf6i2aYFZ0Oq1UXuhJ8APJCoiKgLJy7AiYf kkeithle@dhcp-100-19-89.bos.redhat.com", "key_options": null, "keyfile": "/home/kkeithle/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kkeithle", "validate_certs": true} 2020-04-10T06:59:29.964 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de', u'name': u'soumyakoduri'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1q3k4RvLLCaG874GDJjja4eYqFwrMjsTU+aRzambaVuCmAJT3KQMGuwMjTuCIN7R1Fqr+/dxUClamsfsGqPMe0ASHT9X/0AyNgEkr6na1Fvk9NAwAFelwtiRIHhclzYf6KY8QHZN/eBy43prVKDXd30b15gwQyVqU0/xmny0GrYCN24k3N/R3divt2GVqptYgEwS2KEyFT5EMnJWjdIVG9mBOHrgec2VF/rc6w3BZ0s56iKdpOeCgk5ISYTcom+Xrf+fSDHirlviZv05tBP8ESmampUT/EnwF8Mdaln5XJvZPhrq2JpUisi9fVeR+zlzg6gRGubvokdA3AERrhCQ9 skoduri@rgw_vm", "key_options": null, "keyfile": "/home/soumyakoduri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "soumyakoduri", "validate_certs": true} 2020-04-10T06:59:29.994 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130', u'name': u'emmericp'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBYa5uEOlFv1s9K9D1SbcaApLwX3LBwASSnRXDHI7MtU paul.emmerich@croit.io", "key_options": null, "keyfile": "/home/emmericp/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "emmericp", "validate_certs": true} 2020-04-10T06:59:30.107 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a', u'name': u'kyr'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPeuSTD7kv8yQX6dW+7Vpq1mqrcr8JGubPYa1DpGTV/gE50d5L42UspUN2FXmICDMiV1h4Q9WQAFma3168f1I5I94qFkV+/56szCMoILClm7IW/EV4t8fTxoyOpEa9l4qSU3qgJpDbY04CkiyVJFuQj24Zq3gD/70jYgGpy8RfA0BihSR6F+A0WZLh/fESjqfor3/pFWhUCRnPEqrZsXyuB8lRgDTyGp4s1cHyUqmFMuCPDRMtok0rSVpaRZj2Zj8lboah19UI/L89ptQgEdqsOKzlkxiT+1GdyIY+jpPCM0a3yZz75l3Qiu0k6q7qUV8T0hm1GZiBguvST1KMcwnt kyrylo.shatskyy@suse.com", "key_options": null, "keyfile": "/home/kyr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kyr", "validate_certs": true} 2020-04-10T06:59:30.118 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0', u'name': u'mchangir'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCj28V9sMybkAzhkiaNiDDr8TfNVyr5DNdbOqszdGEVLy1Rxh/UYnxUclsR9n8lt9VnCft3M4NDTnVwlxOHE8+0oolJE9HP+DQomsgj3abP04EGJyOA0X8E7eIFQNLRbMFCD0yX/SNNlYPsS5fPTFhIa/gbBPmDBE07kULts66Tvc0Yo/2yIfwV1DV34bcWWVGi01XybIHBl9qbvlfBKAUrFEcv3ZdpHNhuZw6IkZY8xoYxo8iF+toLYeGX+gFxqCZUUOO+NBVmb0hVL1LI6bU66IrMOQDkFurqpvoraBTJqwZgXvMPceFUkcVIH4Nm7cK1X2WyMtnSw2XBpLMT8C4V mchangir@indraprastha", "key_options": null, "keyfile": "/home/mchangir/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mchangir", "validate_certs": true} 2020-04-10T06:59:30.246 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849', u'name': u'sseshasa'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeERrQ0NVXhuTYzI08rlKRbHZVFXSv1YC7sGq6aCWRbTCv7Ad/t/3OJz3vfcd2TjBMvABfrYCFgTFWIaJBaeA3BXUUU9y9TwnzuxKv4C46WJYoalVGyLEpWpcGbJ5v+leA6Je/+lunenplb7TBKGE/YTgggEXyX7KnunYlv+Jdm33TmAZTcHrLk5QR/6FdRGKmdlLna0dXZc3v04+AI8W+p9ghAv1+e8ENxJDTF8p0MawBiKWcSj2mysOhQ2Cj91qZ0qTnLLeu0CPf4rBO+qjYIshiCuTLhYt2UazlzME4nCCsVqK7Nz9olA/lgvJLRaPxgQQ7iCMmIZilS6B4Y3oL sseshasa@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCo0zniUEXuQ4JJuuRtfnEK8WHqr97AmXEniGPY5U23MTxhMuP8CSkGWkUHdmXw0H1tDKbYgk97M6TspJropNGKljla2H1/3e2T1BOI1L2WfcsOh6VPYeyOj2ssMJeEDdQgd0J3g+4oj2QS+guPlT24UhO0CBhRZ6waVlJMgagDYyDLCLGI4ASoBYx0AOaoyjnEQtRd/gyejzv7jntkHc9YNBPsJ37X6hTYNA/s2ORJE7QMXd6xI60mTBuCdD3JcPYoUSvyCUa7rIrDc70lAcW5XtxewvMBogTZJLoFG/d6Ugij49h4AfwrtoUCtLyJOWJX6PTfomXely8Ym6KpN/3981+m70dhcHPDJ66QCWMd1LAgftVXTIsebA+Bxfg4Vsbrv7j1lJXzmMfsOl+1wVPe+Y9h28GyeIlalW3laURZGV7dhexGZ8YNxPuzxtAlWtfw+kddbjw0j4ODtgVXrj4SDxqmTTYYuMBkrQ3H33sOgPLx8DyA6e4pckb6lOOEvb41Yi5tyP7BIJlYN2PpNVDkSQd/t5Q0di4Ct+3vqLRCkPuJiJAUyaBGT4HM7yQc25+Y0B3sXNON7BjeaBXB7OwKZMp9w1jwbJt4JL/aGOw3HVljVyVgu+pIuQ2ISd9VBcmAJK5EGdWe5Q57h5fJkD+TVrXgeNTaufdclh5vWOH1qQ== sseshasa@teuthology", "key_options": null, "keyfile": "/home/sseshasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sseshasa", "validate_certs": true} 2020-04-10T06:59:30.247 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286', u'name': u'sidharthanup'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC1OVCo0wMITAp8PNUqsTRPFz0SuBL0r1RzKFJUoxQm7gsP9qttZUQRlfiuZ7YRzkuGLOEBngPuIcV995o0lZ0+LVHA8jxWEJamb6sM9YgnonbGKDHfl4lT8LJd4YtjcWnMGvUUdPiPgIkfrpxXtx+RzJ7eCTmgRjAJziZrRh3AqXuKty+s9MDnmK8EQ2smpLdkMtZG5W3U2PiVaGgq2FIIOAoqr58xGqFgZ4tIpvWFpNxVsDZXIg2NdC+ouuKQvuJRWq9eERRHwq26CR+kDFNZeYSyhYKxNBqauLCsAoXNFCJzrdX5Oi9Ox4lJTsEtYX+SMziVipQS54F7MnRAUccukQ86IbXz8C5Ok2EzNo6smwRZ6UPKRvebWjIvfbeR/iwPRnrWcNgjGcaTR6yhlRJh8pBkW84nyY/bJa4WXsoNLgutdsYkjl18KKhbBldjdf0ovfL7FVgqYfAo/wpQYJJeMebP5QAczxvZIegjHRrAvbs/nbQTmuAkes3WL1hhBjSTwyIbWCzAuBJk8MNTmeZkVBvub7XoIyIySppfoJC/1a+ZsxGkzAlgrIxXPYrrBzTtXd/9YRiIgCYx3UtfdbPkE6hT6GblH6ST6QESfWyyvfGxwlOmu+F+icRyVVJbgM9T/PD2W6I04u2/1mcfoHwaQGdwvbbBkbjWzm+zwNSisQ== sanupkri@redhat.com", "key_options": null, "keyfile": "/home/sidharthanup/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sidharthanup", "validate_certs": true} 2020-04-10T06:59:30.390 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189', u'name': u'rfriedma'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIC6uqbIg9XaTlDZd7KCX7FXSzfGFMEWXhx42ktz3ehOPLviA/yNpVd+LmYHSgzd5Szd3mMxleisiE+JdSq/q5YyPAF6eD6+wfynkpMv5aWNHGI0yVfKbCwdOX8bY6yqhyVwMDo14uoFvdDcM4Lj+vz8F75VJHUU5ihJNOW24EllobTLzR4XI6AcM+mjbXRnBLLVoBSuDaU2D0kWqMn8rDCJKdUJnoiy9oxIok8eMLfGkGsL55oOx9BflMSz/0jM9X5GLQahYiECbXjvCIarCkUFE8txNoXM9Djmg8QqhAQDhY0hzJTauFJJaD8Ax1uFQlXONed7tv7a7Tb3dH+/jb rfriedma@dhcp-4-239.tlv.redhat.com", "key_options": null, "keyfile": "/home/rfriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rfriedma", "validate_certs": true} 2020-04-10T06:59:30.412 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31', u'name': u'varsha'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCkeYXWjU7wVdODC32U40QRFgFH7ZEuEsOeWWaKx3Y5EBuAcwkTHiF9Bq/2mrDu+c9W46S8L38Ogc9GZ0DcCXtN/Z4z8iKOIIi5U1GzvwdbKuiAPCd8bdYP0vo0MOIRgylJo5hcnB5BCwCXRN1hmci1ES5UbSeQg0rydMiyYXHZ64AT47+qVUpn/mVP6PVw00pPqt16l9YxhbPFG74Y/WnY4FyrqXGQJfv0tcP7+UR7GQdFiI4QQV3ZUSRHRLWGcmoiiOmGimLhtzEmP4Jzy4gF4wg9q8jyX54NjfxDRZs+YuaoqqWKz/HSz5ic7W2RGKHrRby2N9wNu19nmC1NzIn3 varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCW58OUR3pniaYgvr9xAExeNm4eMxgaJfFbEfEYPztTWv/xxqqihBYCeFltfgWbu7ERTy+hVNo1PJWyyczjl5Ivc3FdRmhykosygs7X74PPh02c7Em4MsSTr7SSPNN20rkJbWqiYMMOBbqbiZ7JXiSUNVilLHgZqZBEQc5Rf5I1PyBLHHJ2ZruPKnDPHCRLdwJ+MdXy2haKmGrQLhtzIKaDzh8VkfojL454k0aHCIh34Z0oBdP9757mNuYJUVEkKZF5t2oML84lWGMgwZxUWN2yO9ALTti4tNH1fXA6NipS9ztE+3hEJe1fWXEoUQDX9JNu4qEHTeCz0secQMkyZVDZSQk8yDUQxQ76yiett37sQB1WQR/3fclcahmqQ4dUS7M3Onyz6k6zia896PBK6G48TBUbFa1rw5Hx7WfNZluDMkv4YxkQvfz7XVzZx2/lhs45Kjh1UR9n7LaW/NID98vokcvASw1N6Bk/6pOd/45iPyEJemwovnQLs6ecwlMIikM= varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDcQ8xMjwVf+KiFIQARcbFT+wg4NHysBRrFxk3Bx5X2k2hGKcEYR2tY/ApHZYfJ6me3LUvJEXg2c/6xa6ibVTUp/90l4y0cF6RmF5uB45J0xuBYoFJd8xJkVrhDIlZix4aFfb7TrmJEqjleBrkTwam7Q9aE3lss16nVOxU4hiViYiCANJ45wobK6tz7y5btfzrHE/iYpwniDvzLwK7EpkEK336VVd6f9wV0vG+ErEFxnNVxheXLLm05LuvdVQvFqJcTFrc7/GZ54yKg5IgwVwXqSAyOXIhgduUqk+oN1ciA7jnSqdiOKQPS/VmYkbhw2YP85l8wGVreIYUmOG19QEyVLB82oB9KS1LezM2KCH61XW22+85dHf4PvuL9DULhGt6wBCvpO10OHxw0JvHXbEgBPhiyd9j2yJmcQozbRPth/sMUvLKNXOGabZhhkPBYL7xZICHc+UwxrXs0h/j9jOjLyHzBAWMomF2lzgbvtzjcbxJ1a5oclJ6IW8yHJllkOh0= varsha@localhost.lo 2020-04-10T06:59:30.414 INFO:teuthology.task.ansible.out:caldomain", "key_options": null, "keyfile": "/home/varsha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "varsha", "validate_certs": true} 2020-04-10T06:59:30.563 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n', u'name': u'sjust'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6RDgqwA0GrvE6KqRaGCgYnFhnxdi9ehjIWvfaMa1oq0CqYxUAee2bNbp/X2Bd18Vqk8iH9fpE0+qrLgAH/8BpD+MtgGikCFMORvazeUtOCF/OCDO0AMg1jvl8aTdvrQU8ZTDPXa65Avpkc9vwugcoplnwrW7lKx++XqCWsSlfHgEdyMV3wrt/ZDFwI2AS2bZBpWM6ViG1KSjXjgTi/T3hSZPY+buYvWjzoEjQO0946iI3iiKKCGohEpCUnG02oJXcS0vvgl8dWE71451lbainEKJ4hbDhfDmy57s6VXfVUANsczwiPWiSC/ZmMlDaCCBeKHO0Gz0xXaDXd9j4SA7P sam@Pondermatic\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYylAUS12hF6XndDQ3CkrUddKmXNSd8pOlHwO+pWaDTJPKxwtPL6sM3aOtaAPLrGkv1zD2k3iZaxmDwfPYGaXOyb0KS5ZCddtOyNXj5JLrjxh2niqFPP8SIkdHLOP+0Hv1/tB0Jz75RV+tNIT1SJTm7OZ8wZNy0qYycITSooJ14iEgyZ7rOa3EEzbtIVpHS4+YqIuIJdV2oV5egCrJyRv4sPUrS8ERXD4QZSOlHE+W4Tf+WVrEfsHOdDLRlAXJUzhuqgWtVouiYcW5ex294z28eA+12lKaPQpE0cft/NJ6FLs41yqh4ZN588bl8Ada+yTXEaHWioQEEJMlca2jIgrP sjust@rex004.front.sepia.ceph.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC08RrDQ2M7Zjcpx7RL9e+tjv48kBgDNyTenX1QdxW22JA2JR3D3ThA/ej9HtiPWcIVz16oZ4tzrCEVwaCMfb 2020-04-10T06:59:30.564 INFO:teuthology.task.ansible.out:smXq2z08ZXYqwiBuoXJVshU7lIKVqAtadT4F6Q9eQvxEo4zWdBib1W0/cqNLlYzU/DuwlUqYnWBDLGxc4fD5Kxrz366oWUDB/ULCpl77TMqfTKlStCgGXot0zLqiscqzq1W7XsuA7A5vQw/JTq7MvIME+SkUuGWX0WKwTMIRZKNWBGmlf51bNru3GmmAnKBUFsJv9H4G0FPqGY/aH02rr5dAgTTmfKt+XbWRfMTMpjJqVRGX03+1mQNoYQ3kNDz0wl sjust@office\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA5UMPySf/h9r2/OryRXrO9Qq9k5J9Dn9nvJYyaYCE82H/qkBJGboAJgyIXfbsGYO5uPM25d3fGRH4magzAQm2hGWNbbt1YWn9HaIyKCCXo9JX600/dv3OZbHyGGmgXPa4Anl16j1c2vxRRt2+ArjlyyhP7cAcRkilUGdagK9u6ALrwnz9hhXYV+BmXrxMm7zZCmdWHcWz+YxoA65es4+w+aLvVZWmUCZLF8uZrcpPSw+oK3SfZE8JPRc5vWulXbWEEfIoHb9AszPIpSyzbfPWXelmc3XlG0KD66t7w+AnNlQPt3JVePRCfFHPYeTj9B0k8sxPyUqTQIl5pu6GppRMOQ== sam@Deepthought", "key_options": null, "keyfile": "/home/sjust/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sjust", "validate_certs": true} 2020-04-10T06:59:30.585 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b', u'name': u'lmb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIXHA8BwV4OnqlaHHyYofsp5X+wanose+aneJVx2Or05PcWA9c7xFEOerkvEn9LvivSRd9G9o/S8SvnGH91tG2bbOAidVZ69FGul9OKa/BCJYUP0vVBX09/Td3n7+R2lVlvu5XiyC5SUzCjaQEZLOpjq23rqaqdCDXE6wFnxfpkmcsccbG/xUiOEBeBGI92lntdx99Sn6CxWWA27lMMaYkCRvX6ziRZEm6O44N2nMm+qYm0jmMyGv1iyQkJxtt0YaLPp+X9t2trUEzEjgBCXqqzEzBjB7T4rbzhAuvU3NRHiAoWXSnrY5AQOTTvC/wpaZV/gF2NDNThjMcpcEdNstleOctXtN4XTTJQewbG+t7I9knR00TPpsoUvUtpbcoFOluGgWTDTFa6GCmDPppx23nZgulOniZUVTI2CbScSpc8sfp1JE7Q6vLECY9lJ+GpVCbDAbIEE9WHXSkJ/6L9yf+ZoHbpqCkEFDcWLfz3B5NFoJ5gne8B8uzd6ptKVtJrP8pMvoOyydyvQlmkgh8tNDsOnp6kaUNG0NHmtfGlCTkbM7oKsKbf87Cv1HgYuZl/HP7go5sOX02LfBfbvWrawZzOaZ3dewEq74w5hVMKEHrmJPstIpcHSxZpq33f2j4Z/uKW9tJat6rgSWK1I+T3tCd++76apxbOYl5TDOaoxCyjQ== lars@marowsky-bree.de", "key_options": null, "keyfile": "/home/lmb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lmb", "validate_certs": true} 2020-04-10T06:59:30.729 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc', u'name': u'ksirivad'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMDUcToIhs8tiIozgPudiYyD2upEX+VvsiJRti7zrA/1HPIDARYt2+xyWCASm5x5vdohRM7wnBLUlbBVnvr6ZUQvfShNDsS1Uu7hEfYF38MtW+dVwvGlm14J4m1gbjzqVlZvO6bg6rfGo7Qer0bKUlxtqbisQ/kCg/lVWR04CNxdRtZxEpaQ21y4C5jAlrpFTzxQSuiCLEyY3EAKdZUaZElFEcXlna0ysCP8inlzhF402PkSdl9DkkPpg3IR2dUEGqsblWSdLVk19EFavjtGAAnsvaBWWm2rCLNC5RmhGQaS/Z52Aes1nBuT4cI6qLjbEMqAzLwxiOR7l4IPMpK5EcuRK+FaqQG0wMAjxPc95CmxDzi58ZRJ5ERV8vgps73F1AJXWU9Lb1P77BTsLXqiwko2mvPf6e0LwCxQuxYJWc+xvO7Ji011B+nm5eLCCC/LcDtTPjkimTh7nkvUNi/yTKCXi9KwLISAHHd335KnfrL28rU3GahMXs4NjOxuN8eKM= ksirivad@ksirivad", "key_options": null, "keyfile": "/home/ksirivad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ksirivad", "validate_certs": true} 2020-04-10T06:59:30.747 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'deepika@x1carbon BgvhtHMrCcelT98ekPDMoA 6fa3cf08186b93b966b68841f3507a0809d348c5097b39b6140832cac2afd511', u'name': u'ideepika'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon BgvhtHMrCcelT98ekPDMoA 6fa3cf08186b93b966b68841f3507a0809d348c5097b39b6140832cac2afd511"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDj6DxTZNodVnwJZrnSZ/jv9Cw4uYLb8MGpccmOvtwl93sVBvqq597AYnBqYiV4VBbKU3GVFbT41zkplia1iCfXoddttCOyWmEKviBXRnAsd5f80dxL0pfzrBJvCaH4NAfYgVw9JFWsvT981b+nB6vum4Vz8Dw0PM5GPnVUjsZNFkpmDPUBL4yHPmMCGP2WtUurJVD3Gm9t72XlG6Dl/vB9Qw/i4IY1SoCCbuhhNgE7IN8JXeGmVJbRwDeUlP+Q8Ti7iedV7YB/W1opt6eT5EPzX8NcxTc0c7rrDgsUWjFQYRSup0GLK9ofdHs0E2RbPPO5OSc5Z2+B1IEV7fZeyJwAyHwUgwx7sSGy7aROdvinaj/hOBglcgISyQ2CY7LTV8NGWJquo7yRCJXhZVIHfgqfBcrgWdNbr+vNBXnWe5PCtLhV8ryHPSJ/7Ru+7voQxgvwux5SdZwxXTbWDKni1f6/8bNrhRx6bmLK9xuYAgt2FhlIivI7L5kQF4etcBiaf68= deepika@x1cabon", "key_options": null, "keyfile": "/home/ideepika/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ideepika", "validate_certs": true} 2020-04-10T06:59:30.892 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46', u'name': u'gsalomon'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDC0h+TpaUkm2V6Hbf7aPCTys/nsm51CmY9s+bdwEXAiSAWpq/7Xi2LlN0mZgsrAYBHbr5L+Pz8x0YdigvWysUK6U9XRSPM28cbWmh8F25re9HEk8ARESkMZKTv/hKFs1v7kTHbT9hNNynbQGQOqEaazjr0shcli+Byvop27riX3QDDSsd5xfCjjyDz9fiA9YESlF0kbXyFUKPsoaBhRv9FUhtH6DxlygxtxTsKyKgeoiKw2sCfeUMckcJKnGCMtkXW4U8h7NL/fPZFyBaMDRi/TtoBKaVGLaWqF7RuukfjoMCwhuIDmuTG3WhanH4HJrfNenPZkpZUBhzWwZcleao8SD3mXXfZ9gcZb8QB2ymEtTqrAEyDoNWO4q8ljNP3WJXIgF0h0Sjmx/CV/i0ArEYlBMwBD7DFVnPJqCqSyypuQdDGsDNQZxaRCiYg7nIslFOpDr+0PLp78m+CHyYbSFLTvVVJoJ1k0YhMnDPHM29nt7fMGx6Qrigwnuk0yKg1bfgJ/t0TOqEw0bqGKIvHnyoP/n0ObazEsiPl5pRr2ehs/Rqf35Iwhc1i02Hhz/3Sqwc1eAgqm4/OVghxqUJGpRocua5Gf/vO32wwoyGxh2sl7y6K2pNDA3glhKYbZeIh1fjnaTjikgleqIQkCBxbTeAE7n5+T6TcFtnII6RSxYv3tQ== gal.salomon@gmail.com", "key_options": null, "keyfile": "/home/gsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gsalomon", "validate_certs": true} 2020-04-10T06:59:30.893 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4', u'name': u'ofriedma'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0POsA+0NyxecLo3Y821G6W3+jJPKFMuLSIl4FSLi25vuyuM5+mFGmCNWv8gg93eQhG9Y3SOygqmkaxzxDtBS/V6VNC2f8hZsVw9B+HbnfggvCNQSgjJ5rhvf6IHfx/aHq4IHF+stqdcoWhwR/68wIhGNqtP7Nhqnzk2aN1kG9bU2iVk4CH8XGIwqwET3aWNuCMcHi2AhrvDBpZu7bzo/6ARUJDqEfTEC/XIkzv8/mEkJ5igcycARVy6dChuCwfZc8rhfwQpekwx1Y9/WBFdNQlYkN4fVe8yIqT5VsQ3AcRv21GJk8L7mwB4hF9dVAi4p1cSyxatR3kDifsFc8VQk9 toor@localhost.localdomain", "key_options": null, "keyfile": "/home/ofriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ofriedma", "validate_certs": true} 2020-04-10T06:59:31.052 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52', u'name': u'h4iresis'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDc/8sWrMq+bqKBPmDPplUgdMoY2SQL+57jRSh27PAoYW9rNVR/UgK7EA9oxFLfoi9wJ9+VKU4opeoohOXLWQXXatLvfXjGE3fRYIonQfwJqKrasqQJTGWzRZ9t1/2Cb0VHkoAvipCJzWmgB46X6q73XW5Td1H3b+AINd6E4Rsm1QjBOhk0DMjCdfwrVUZIfGdsEo+5u/AwioHwH+NDJvXTWT8ZigeZBNHJSY/EQn1+15ut0QB2rvXkJasvvDlVbEsPQtM4C3VX1ronmAnKMvCp1O4EtAXaoKSOx7XEN++UAvVn7czDCWk/UFgpzZOqTcQJwg82XtS4sZP23CpNe0G9 h4iresis@fedora", "key_options": null, "keyfile": "/home/h4iresis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "h4iresis", "validate_certs": true} 2020-04-10T06:59:31.082 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de', u'name': u'soumyakoduri'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1q3k4RvLLCaG874GDJjja4eYqFwrMjsTU+aRzambaVuCmAJT3KQMGuwMjTuCIN7R1Fqr+/dxUClamsfsGqPMe0ASHT9X/0AyNgEkr6na1Fvk9NAwAFelwtiRIHhclzYf6KY8QHZN/eBy43prVKDXd30b15gwQyVqU0/xmny0GrYCN24k3N/R3divt2GVqptYgEwS2KEyFT5EMnJWjdIVG9mBOHrgec2VF/rc6w3BZ0s56iKdpOeCgk5ISYTcom+Xrf+fSDHirlviZv05tBP8ESmampUT/EnwF8Mdaln5XJvZPhrq2JpUisi9fVeR+zlzg6gRGubvokdA3AERrhCQ9 skoduri@rgw_vm", "key_options": null, "keyfile": "/home/soumyakoduri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "soumyakoduri", "validate_certs": true} 2020-04-10T06:59:31.186 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5', u'name': u'shyukri'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCqhWVTosH12864VGY5S3/k0yKc59GbV0m2WxD28gAf+uvJ1/y8gekD2wDzBv0DWqEekYLV7L68gpapDXyGosQeZ6n5OO/NZFZmW+Xq1P7P1QHJil6kVpwk927lzX440pwNf3+PRvTbLzlbuVFWJRpiykv+juJyIFjzXLJRF67wB2qgZVm5cRs/GOi4fnvTaou/Ae51mvsO8M3JL5cphApFKD/AO/X+3pHBhLjvaW33cYsp0Z4Rce1nm8wxSmvuJRF+HAfjBlknhDl5ZaJGA96RYA+g9WbAXimFprg4xeiRp7liPExQwtVgackJweoC4MahC1glmjC3eOh9dvrrxLnALOLMOkyCSK/tOF/iq2PJaLViObanPI8mnNkMLbdJyzUBf8hRGdSLcw98qxaf0sSZ+vz4VQS6e9CwX7ikkNNsTeYXjnJ+1yU3mfCW1CXJDr7kBDVG2OTLefCpMY8WBAYDiEaK0RW83cBcFIKfecKYd0qStHRaOsvmyWdFYgBh2vM1UWjro5WXlb+7aJPrUp8ZruapkUXUwhjHA7FwGA4ysE0tsOVVFm0Dw8tvyVVkVNm/9EObG28dWctioZ2+ZEzTWgMJU26Q+wOJ/U4qZyIvpMRUgPZc5nM9J8kcmvNWtb6DV9k/6p8h28kiOxPkxLtPP+3TlBCHjq//mhEnJZ/d8w== shshyukriev@suse.com", "key_options": null, "keyfile": "/home/shyukri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyukri", "validate_certs": true} 2020-04-10T06:59:31.218 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a', u'name': u'kyr'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPeuSTD7kv8yQX6dW+7Vpq1mqrcr8JGubPYa1DpGTV/gE50d5L42UspUN2FXmICDMiV1h4Q9WQAFma3168f1I5I94qFkV+/56szCMoILClm7IW/EV4t8fTxoyOpEa9l4qSU3qgJpDbY04CkiyVJFuQj24Zq3gD/70jYgGpy8RfA0BihSR6F+A0WZLh/fESjqfor3/pFWhUCRnPEqrZsXyuB8lRgDTyGp4s1cHyUqmFMuCPDRMtok0rSVpaRZj2Zj8lboah19UI/L89ptQgEdqsOKzlkxiT+1GdyIY+jpPCM0a3yZz75l3Qiu0k6q7qUV8T0hm1GZiBguvST1KMcwnt kyrylo.shatskyy@suse.com", "key_options": null, "keyfile": "/home/kyr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kyr", "validate_certs": true} 2020-04-10T06:59:31.333 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e', u'name': u'scarvalhojr'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxDJtmmcnrbA4AUsmBBZq/M4xNdIbOT+cQQxhUKeCu8F9mht+RznwyaaJbVIlD+UNv9LMhRyzI5oktSalokNltlrtPQMoP52PXH7KFFGnVnexCHAk4nGvEhuZCniO5T+ENm2wFR47lFEQK7lyJJLp5PFiSM7vM1MK6UkMvRmCTRildjaYLyf3EEvBXifbsGmR0tBIXtml6KITCAaT6l0E94055oEGN2G5NhZIvfLcojt5liEkXEHQNdfkUDXM0Go43JZzgsSWDXC/7RYLrAJwz5mq4qCa9YZL/3Spjso37+1oLq7lOQ0TDJUAQXmnW0So/NkzECx4MolVcylhTzmgj sergio@milestones", "key_options": null, "keyfile": "/home/scarvalhojr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "scarvalhojr", "validate_certs": true} 2020-04-10T06:59:31.334 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849', u'name': u'sseshasa'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeERrQ0NVXhuTYzI08rlKRbHZVFXSv1YC7sGq6aCWRbTCv7Ad/t/3OJz3vfcd2TjBMvABfrYCFgTFWIaJBaeA3BXUUU9y9TwnzuxKv4C46WJYoalVGyLEpWpcGbJ5v+leA6Je/+lunenplb7TBKGE/YTgggEXyX7KnunYlv+Jdm33TmAZTcHrLk5QR/6FdRGKmdlLna0dXZc3v04+AI8W+p9ghAv1+e8ENxJDTF8p0MawBiKWcSj2mysOhQ2Cj91qZ0qTnLLeu0CPf4rBO+qjYIshiCuTLhYt2UazlzME4nCCsVqK7Nz9olA/lgvJLRaPxgQQ7iCMmIZilS6B4Y3oL sseshasa@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCo0zniUEXuQ4JJuuRtfnEK8WHqr97AmXEniGPY5U23MTxhMuP8CSkGWkUHdmXw0H1tDKbYgk97M6TspJropNGKljla2H1/3e2T1BOI1L2WfcsOh6VPYeyOj2ssMJeEDdQgd0J3g+4oj2QS+guPlT24UhO0CBhRZ6waVlJMgagDYyDLCLGI4ASoBYx0AOaoyjnEQtRd/gyejzv7jntkHc9YNBPsJ37X6hTYNA/s2ORJE7QMXd6xI60mTBuCdD3JcPYoUSvyCUa7rIrDc70lAcW5XtxewvMBogTZJLoFG/d6Ugij49h4AfwrtoUCtLyJOWJX6PTfomXely8Ym6KpN/3981+m70dhcHPDJ66QCWMd1LAgftVXTIsebA+Bxfg4Vsbrv7j1lJXzmMfsOl+1wVPe+Y9h28GyeIlalW3laURZGV7dhexGZ8YNxPuzxtAlWtfw+kddbjw0j4ODtgVXrj4SDxqmTTYYuMBkrQ3H33sOgPLx8DyA6e4pckb6lOOEvb41Yi5tyP7BIJlYN2PpNVDkSQd/t5Q0di4Ct+3vqLRCkPuJiJAUyaBGT4HM7yQc25+Y0B3sXNON7BjeaBXB7OwKZMp9w1jwbJt4JL/aGOw3HVljVyVgu+pIuQ2ISd9VBcmAJK5EGdWe5Q57h5fJkD+TVrXgeNTaufdclh5vWOH1qQ== sseshasa@teuthology", "key_options": null, "keyfile": "/home/sseshasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sseshasa", "validate_certs": true} 2020-04-10T06:59:31.581 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b', u'name': u'rakesh'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEOigBGMeElb3uHXX36HxLtPJMG+iG/6D7NDFCVR6AyYsmimBQb6SlSIR6SaFULb6HT/IFVk0Jf/z0HoipAP7Xu+wvP/J6LFoMdguxylh1ov8raqxQJrGLl9n64EI+qWINwEM0MKMUnDQ3tmMKBl78LKQVQTYBx1+9cXsT6DE1yhgmV79XIvOOb+v53agSi/kLJF2i9K04kpi1OejXg95Ub2eLMY69Jm8PnLy1Jq8DhC8pz7nrfzbKtOC7HbytTAO1XTlb90lVhqC/1yTMAUc/rpJwoSEHc2epGeZWdMsJhqhX2EqMwcV53M3qcFSGreGNJ/Ncd4/JHxz7q0IU411t rakesh@localhost.localdomain", "key_options": null, "keyfile": "/home/rakesh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rakesh", "validate_certs": true} 2020-04-10T06:59:31.582 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189', u'name': u'rfriedma'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIC6uqbIg9XaTlDZd7KCX7FXSzfGFMEWXhx42ktz3ehOPLviA/yNpVd+LmYHSgzd5Szd3mMxleisiE+JdSq/q5YyPAF6eD6+wfynkpMv5aWNHGI0yVfKbCwdOX8bY6yqhyVwMDo14uoFvdDcM4Lj+vz8F75VJHUU5ihJNOW24EllobTLzR4XI6AcM+mjbXRnBLLVoBSuDaU2D0kWqMn8rDCJKdUJnoiy9oxIok8eMLfGkGsL55oOx9BflMSz/0jM9X5GLQahYiECbXjvCIarCkUFE8txNoXM9Djmg8QqhAQDhY0hzJTauFJJaD8Ax1uFQlXONed7tv7a7Tb3dH+/jb rfriedma@dhcp-4-239.tlv.redhat.com", "key_options": null, "keyfile": "/home/rfriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rfriedma", "validate_certs": true} 2020-04-10T06:59:31.625 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f', u'name': u'ryg-'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvRwrsN62CReT0/kOeFEXGA3THhzJ+hwZVpUzo7VMRZeSo6LgNlVGdrQchMHnlwBvuwaVNJlBAkQr+5J5BB3ikrmbcnLHfPutXBBBB6uDfSwwbqjzUY7elTbz9ul6AkegR9K65O44ouD9+WC76D+Tp0maqnP1w0O3h1GVxDHe34LpEwBvZJ+/BSYmBpwRZqeXFRADORPzdtW08PSiRErGHb8j+j5i+br37JfOOh+A/9dHwQw9alBVL2DijwvrPMa7dV5yX00dlMSTL2LLo9IdI4aFPmie8oic0eanFbQDq210ilOR65IGeshKHEKOq0KFFz0nSK9OdO03OkSEN/0d5 ryg@tapir", "key_options": null, "keyfile": "/home/ryg-/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ryg-", "validate_certs": true} 2020-04-10T06:59:31.636 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b', u'name': u'lmb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIXHA8BwV4OnqlaHHyYofsp5X+wanose+aneJVx2Or05PcWA9c7xFEOerkvEn9LvivSRd9G9o/S8SvnGH91tG2bbOAidVZ69FGul9OKa/BCJYUP0vVBX09/Td3n7+R2lVlvu5XiyC5SUzCjaQEZLOpjq23rqaqdCDXE6wFnxfpkmcsccbG/xUiOEBeBGI92lntdx99Sn6CxWWA27lMMaYkCRvX6ziRZEm6O44N2nMm+qYm0jmMyGv1iyQkJxtt0YaLPp+X9t2trUEzEjgBCXqqzEzBjB7T4rbzhAuvU3NRHiAoWXSnrY5AQOTTvC/wpaZV/gF2NDNThjMcpcEdNstleOctXtN4XTTJQewbG+t7I9knR00TPpsoUvUtpbcoFOluGgWTDTFa6GCmDPppx23nZgulOniZUVTI2CbScSpc8sfp1JE7Q6vLECY9lJ+GpVCbDAbIEE9WHXSkJ/6L9yf+ZoHbpqCkEFDcWLfz3B5NFoJ5gne8B8uzd6ptKVtJrP8pMvoOyydyvQlmkgh8tNDsOnp6kaUNG0NHmtfGlCTkbM7oKsKbf87Cv1HgYuZl/HP7go5sOX02LfBfbvWrawZzOaZ3dewEq74w5hVMKEHrmJPstIpcHSxZpq33f2j4Z/uKW9tJat6rgSWK1I+T3tCd++76apxbOYl5TDOaoxCyjQ== lars@marowsky-bree.de", "key_options": null, "keyfile": "/home/lmb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lmb", "validate_certs": true} 2020-04-10T06:59:31.775 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_key 2020-04-10T06:59:31.776 INFO:teuthology.task.ansible.out:s", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2020-04-10T06:59:31.786 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc', u'name': u'ksirivad'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMDUcToIhs8tiIozgPudiYyD2upEX+VvsiJRti7zrA/1HPIDARYt2+xyWCASm5x5vdohRM7wnBLUlbBVnvr6ZUQvfShNDsS1Uu7hEfYF38MtW+dVwvGlm14J4m1gbjzqVlZvO6bg6rfGo7Qer0bKUlxtqbisQ/kCg/lVWR04CNxdRtZxEpaQ21y4C5jAlrpFTzxQSuiCLEyY3EAKdZUaZElFEcXlna0ysCP8inlzhF402PkSdl9DkkPpg3IR2dUEGqsblWSdLVk19EFavjtGAAnsvaBWWm2rCLNC5RmhGQaS/Z52Aes1nBuT4cI6qLjbEMqAzLwxiOR7l4IPMpK5EcuRK+FaqQG0wMAjxPc95CmxDzi58ZRJ5ERV8vgps73F1AJXWU9Lb1P77BTsLXqiwko2mvPf6e0LwCxQuxYJWc+xvO7Ji011B+nm5eLCCC/LcDtTPjkimTh7nkvUNi/yTKCXi9KwLISAHHd335KnfrL28rU3GahMXs4NjOxuN8eKM= ksirivad@ksirivad", "key_options": null, "keyfile": "/home/ksirivad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ksirivad", "validate_certs": true} 2020-04-10T06:59:31.912 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42', u'name': u'zdover'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlnlBzRdWRqns/9VrYuF9tLILUjeGd9z8Tf/dPZmxAt3nnZqpTvI2xXemQpyt/2lsD0U24EuTc1Y/dBsgqvXtZOw51TZE8RK26hvCCaKGfcKONjvR7goHAmkYPHK9+F6m7fgYkRzA0+ZW4HcWNh/xxhhL2G1EGfz5bzIoGQJHSdzc6VeVIsY7HiYEMTxvxdsgV72Kjq97BdmHm56RBzoH24oN0UzNZS8Syx5SuWj9dA3IyFBPwjLlozSk6U2Ofd7qaXuQKl807eD9g6RhRldzHPS+i0EK3EM6Jvho5Y08k1KqV4gIEwMdzDbD3SH0xq+nUPFKcDTUFNp+tkg10GxqN zac.dover@gmail.com", "key_options": null, "keyfile": "/home/zdover/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zdover", "validate_certs": true} 2020-04-10T06:59:31.923 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4', u'name': u'ofriedma'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0POsA+0NyxecLo3Y821G6W3+jJPKFMuLSIl4FSLi25vuyuM5+mFGmCNWv8gg93eQhG9Y3SOygqmkaxzxDtBS/V6VNC2f8hZsVw9B+HbnfggvCNQSgjJ5rhvf6IHfx/aHq4IHF+stqdcoWhwR/68wIhGNqtP7Nhqnzk2aN1kG9bU2iVk4CH8XGIwqwET3aWNuCMcHi2AhrvDBpZu7bzo/6ARUJDqEfTEC/XIkzv8/mEkJ5igcycARVy6dChuCwfZc8rhfwQpekwx1Y9/WBFdNQlYkN4fVe8yIqT5VsQ3AcRv21GJk8L7mwB4hF9dVAi4p1cSyxatR3kDifsFc8VQk9 toor@localhost.localdomain", "key_options": null, "keyfile": "/home/ofriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ofriedma", "validate_certs": true} 2020-04-10T06:59:32.055 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b', u'name': u'rraja'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC4zwWf+kMEDgaSOrX7AlD+UCOgf4/fRiorQVJqC8GfMNynfiFVZaThngwIrwtQROfXHbzspi+AcRvDLewY25mXyuxVgOfSbuSIEqiUNvzcVq9IItI1J58xeDBpIktAsoHHS6JENV56fAT6krzRAQLnSa1k2b0pyKuPBzFSeNgiaQMhjrCLAaCi9yLsdKpnWgXhyfrdqvN2r/Ctts/zlTPqtRYdvdIL/p/l+vg7Q8S3OhuAGtimaEq3RE70xadaPbGhsLISMTrbfYXbxw84KKZpNYwO4zmQ2ztcvC9D7A/8ov3+A5Jq42G+pq7ORuix2NoTWtADMiwvNe1HyzS/7cX/BuXLFssRY1bzRvp+x2ExZ7b+bFx+TE/RduRh0eyDNLg7SyL83aeTr2Usa+mPoCAMKed9XtYbRnUu7ox4wmfeAgx6jM9uG5uQfauJ1XpDddrVBG8KYBvSd0BM2cyRdcAXqwxg9AVPZcdE7sbBk44CGVhw+ziZ3e3cLBTO4y44KRQre+1Bhe/Ze8OTQWLFugl/XagFlJE/L6fGUIPuFojmQvlw5dBZwtUURnxsu8yHm3iIsj4y8nR1zHMj+zuMk6rF3ZW0epPAECAdq4gG8X2wv3HTq0vLBkKpi1KMAmntEtcVwF0fzMt5XYqRd4jceFgonAAipJNDknHgi6yaUzBYmQ== rraja@redhat.com", "key_options": null, "keyfile": "/home/rraja/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rraja", "validate_certs": true} 2020-04-10T06:59:32.057 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52', u'name': u'h4iresis'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDc/8sWrMq+bqKBPmDPplUgdMoY2SQL+57jRSh27PAoYW9rNVR/UgK7EA9oxFLfoi9wJ9+VKU4opeoohOXLWQXXatLvfXjGE3fRYIonQfwJqKrasqQJTGWzRZ9t1/2Cb0VHkoAvipCJzWmgB46X6q73XW5Td1H3b+AINd6E4Rsm1QjBOhk0DMjCdfwrVUZIfGdsEo+5u/AwioHwH+NDJvXTWT8ZigeZBNHJSY/EQn1+15ut0QB2rvXkJasvvDlVbEsPQtM4C3VX1ronmAnKMvCp1O4EtAXaoKSOx7XEN++UAvVn7czDCWk/UFgpzZOqTcQJwg82XtS4sZP23CpNe0G9 h4iresis@fedora", "key_options": null, "keyfile": "/home/h4iresis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "h4iresis", "validate_certs": true} 2020-04-10T06:59:32.201 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f', u'name': u'toabctl'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2mEyxfkW1u84u5BocLCAXESj2qObWkxUd20cmPolhJYlIWLOafa69Dw5A9CjFBp0U6Jx5d0JWcwmAc5ZeffQ7t7vzmtHCZnpKkyZhN9bu6vuCh4N+I643SI/K7tdbB2E8ZE6uI7ORMAan5+t0bNSVolUVq6X1w7dIA5El54K10WNdiwgRKW6e3anNbJ3rBLGEk7gawZqDDOqTvVQmOBGWek3BjSReDvg3rkmTQLxJkiKdxnx4/s7gzH1xTpsS/OludIuSxF0BK064ZcT1t09qcjnT4Os1inV2GfYswt+UhpgqrHqXjEs0Z3y4Oy77QBKL1rDf93CP4a4+eJjqXe+v tbechtold@suse.com", "key_options": null, "keyfile": "/home/toabctl/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "toabctl", "validate_certs": true} 2020-04-10T06:59:32.213 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5', u'name': u'shyukri'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCqhWVTosH12864VGY5S3/k0yKc59GbV0m2WxD28gAf+uvJ1/y8gekD2wDzBv0DWqEekYLV7L68gpapDXyGosQeZ6n5OO/NZFZmW+Xq1P7P1QHJil6kVpwk927lzX440pwNf3+PRvTbLzlbuVFWJRpiykv+juJyIFjzXLJRF67wB2qgZVm5cRs/GOi4fnvTaou/Ae51mvsO8M3JL5cphApFKD/AO/X+3pHBhLjvaW33cYsp0Z4Rce1nm8wxSmvuJRF+HAfjBlknhDl5ZaJGA96RYA+g9WbAXimFprg4xeiRp7liPExQwtVgackJweoC4MahC1glmjC3eOh9dvrrxLnALOLMOkyCSK/tOF/iq2PJaLViObanPI8mnNkMLbdJyzUBf8hRGdSLcw98qxaf0sSZ+vz4VQS6e9CwX7ikkNNsTeYXjnJ+1yU3mfCW1CXJDr7kBDVG2OTLefCpMY8WBAYDiEaK0RW83cBcFIKfecKYd0qStHRaOsvmyWdFYgBh2vM1UWjro5WXlb+7aJPrUp8ZruapkUXUwhjHA7FwGA4ysE0tsOVVFm0Dw8tvyVVkVNm/9EObG28dWctioZ2+ZEzTWgMJU26Q+wOJ/U4qZyIvpMRUgPZc5nM9J8kcmvNWtb6DV9k/6p8h28kiOxPkxLtPP+3TlBCHjq//mhEnJZ/d8w== shshyukriev@suse.com", "key_options": null, "keyfile": "/home/shyukri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyukri", "validate_certs": true} 2020-04-10T06:59:32.349 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39', u'name': u'mgfritch'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCitWIN7jm5KEmB6jqcA+ufKRLodxkJ1GIdaaueCdeXdMHgWNH9FUDCUJk5dcA11MxQO15DRH/cdhb3QHPGiMv4XrBeK6HHYwGttzkcUEz1EDUhpowOrnFETbPa7K39ObmRcgdvoGXOSrNZ4mvptCTz9Xb0MF+Gcg5iJ4g+qZzIVa2/ueecYY8ZorbaH4m/NWUub6Rf01Q04HfxTvKrUFswrbCoE/Aigz/Oh88RiS854XoTvV7rE1tKbyKkaungwMAaQSNnZkq4hi8nbp6pzTxJLV7N/ZYMhVpdxSbWcghFg6cy7aeoCzZMN97WcDhpdfbhRT0dbqCkMmHV6t0usDDnOss2v68UCzkxuA1Hj4uy8rRHjXBtYqI6xsrwCjvmgP05B14TpmFdG+3GdRHx2+QlhnGj7nuPf1KoWW/f3tD85DXoEmVl6RJ4KTrMeEVlOn3kdMLVH0y2hIqPTYMP5cMVNi0EdfeoBdd9kxMKMvoSVBAgmLzGVv17o1lNvVS6V1kyt7gME2PfNR2Np+U1cJVJfUJRTm28mHXBVoDXG9s0zv8+LpQ/gVPS+vsPAfxWWYGW8ZmEV3ZorDOZycSPybjHkPFzDAdUhPG0Db0o+F9ug+22F7pvifsdDZ42zqGGLbyzfIQE3AtyI7LR7Sc5tpN9rKTeSBYJ0yNcDKEj5FUHDw== mfritch@suse.com", "key_options": null, "keyfile": "/home/mgfritch/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mgfritch", "validate_certs": true} 2020-04-10T06:59:32.350 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e', u'name': u'scarvalhojr'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxDJtmmcnrbA4AUsmBBZq/M4xNdIbOT+cQQxhUKeCu8F9mht+RznwyaaJbVIlD+UNv9LMhRyzI5oktSalokNltlrtPQMoP52PXH7KFFGnVnexCHAk4nGvEhuZCniO5T+ENm2wFR47lFEQK7lyJJLp5PFiSM7vM1MK6UkMvRmCTRildjaYLyf3EEvBXifbsGmR0tBIXtml6KITCAaT6l0E94055oEGN2G5NhZIvfLcojt5liEkXEHQNdfkUDXM0Go43JZzgsSWDXC/7RYLrAJwz5mq4qCa9YZL/3Spjso37+1oLq7lOQ0TDJUAQXmnW0So/NkzECx4MolVcylhTzmgj sergio@milestones", "key_options": null, "keyfile": "/home/scarvalhojr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "scarvalhojr", "validate_certs": true} 2020-04-10T06:59:32.485 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'xiubli@fedora1 vosB15bA3uALkBrmjcKFUg 4ef5cb132c30563fd5fb9e4e9352786ad0d2258e41ed63f91c1f65f69b54d50d', u'name': u'xiubli'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiubli", "ovpn": "xiubli@fedora1 vosB15bA3uALkBrmjcKFUg 4ef5cb132c30563fd5fb9e4e9352786ad0d2258e41ed63f91c1f65f69b54d50d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCzVa77kg0qGGE3IpMZgSYJ+UwQiDEOg3Lji470V/sQ/R7g6/v7FZpaYqGn1wG3XeUp2gGACxNKLQfGV9EGU66NvfQYqn9A8dXRpTpZuGWN8EcKK98YrSAq0s0WzeiEzVT86RHzVHIqL6/Zn4B1p9sIeY9Hb+bfKOdtYkRBDoFELtiMRzHPFaiIJFnXl+EhH8wceYqXVNNvCuOUBMnOiDYLDtujvYdPt+ff7rvXuOzbwcAKoINFM1C4FtoUEYriKlsxJ2rr7AblfDm/WIrCZcqRBU4JcudtvZOtHhvN4uL6WgzuzuhIQFyePAeIHFUn6AiwKC/MApq7z1Qj+N3pN/Nz xiubli@fedora1", "key_options": null, "keyfile": "/home/xiubli/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiubli", "validate_certs": true} 2020-04-10T06:59:32.490 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b', u'name': u'rakesh'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEOigBGMeElb3uHXX36HxLtPJMG+iG/6D7NDFCVR6AyYsmimBQb6SlSIR6SaFULb6HT/IFVk0Jf/z0HoipAP7Xu+wvP/J6LFoMdguxylh1ov8raqxQJrGLl9n64EI+qWINwEM0MKMUnDQ3tmMKBl78LKQVQTYBx1+9cXsT6DE1yhgmV79XIvOOb+v53agSi/kLJF2i9K04kpi1OejXg95Ub2eLMY69Jm8PnLy1Jq8DhC8pz7nrfzbKtOC7HbytTAO1XTlb90lVhqC/1yTMAUc/rpJwoSEHc2epGeZWdMsJhqhX2EqMwcV53M3qcFSGreGNJ/Ncd4/JHxz7q0IU411t rakesh@localhost.localdomain", "key_options": null, "keyfile": "/home/rakesh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rakesh", "validate_certs": true} 2020-04-10T06:59:32.619 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f', u'name': u'ryg-'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvRwrsN62CReT0/kOeFEXGA3THhzJ+hwZVpUzo7VMRZeSo6LgNlVGdrQchMHnlwBvuwaVNJlBAkQr+5J5BB3ikrmbcnLHfPutXBBBB6uDfSwwbqjzUY7elTbz9ul6AkegR9K65O44ouD9+WC76D+Tp0maqnP1w0O3h1GVxDHe34LpEwBvZJ+/BSYmBpwRZqeXFRADORPzdtW08PSiRErGHb8j+j5i+br37JfOOh+A/9dHwQw9alBVL2DijwvrPMa7dV5yX00dlMSTL2LLo9IdI4aFPmie8oic0eanFbQDq210ilOR65IGeshKHEKOq0KFFz0nSK9OdO03OkSEN/0d5 ryg@tapir", "key_options": null, "keyfile": "/home/ryg-/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ryg-", "validate_certs": true} 2020-04-10T06:59:32.628 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f', u'name': u'jafaj'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDT+6B5Fr99p8xe9Zo5x8ULcdeA6ne727iVJp/xaguGwGHj6AhuK+R/CjznnHk0yA+6kiblNzjvbnsuzO8T9c8Ayv5KU07MxP1y1pymdYBpEh3NTXmhB6QOTCNTQbT14PL6MVgzK17SqarFCYXoEGuyK9jaBdN9QM76vhEaXc/2Osg+d8boJTkOk7lOPHbzeOCbQU3d+jX0VFOgAQS5OFz3Ug/X7nB0D0LN8bt5FlCZiGO95Iwls/SyPDbVXc9ot8oh4ME7kd7sElokBdYdXPauibRxWKjSDn+hV6kZfSzqwnMFaZ1TC+Av6zNsBuZucNSp3/Ws3T5i9i+21eyS4+qVE8IiX2aVKYuC9afb2lPLbdG2Iy1CqiaeGPkZfiGT9P8iQPOkqae45859UzFAd/XzKjBcmus3lSArKG1EyZl03HQ5P2J60pErCXq8W9mPjxAze2ReLRXJVEgTXnmdhCptcg6PZ4d/k4BEddbdnoX6/BEgFxLEJ64HpTeTVcDXHpipCpxFSRvgyPH/wQGKAvt8AfKBhmeZs7LuyGOhJiOon4DDaShHpE9wTnJ5qlnyg44+u6NCwelGPMAkHKEO4OgcjSsyXm+HYUpxG2s4poloIJzvU6oa3dvJsMZFhcb8CsW75sBYE+YngS/1fg1WU88ZOdaa1yF4F87pVZycpZqY1w== jan@jfsuselaptop", "key_options": null, "keyfile": "/home/jafaj/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jafaj", "validate_certs": true} 2020-04-10T06:59:32.755 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_key 2020-04-10T06:59:32.757 INFO:teuthology.task.ansible.out:s", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2020-04-10T06:59:32.766 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef', u'name': u'jschmid'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo8wE/b2/EDwV018ENfcENVfXSF7vnXAwBSzNVScgxdqr2DVKTWDzhoI0m5CXTqI2HLnJVFDTgN3t5/fFcmwmSAP8ic8wU1JznSPZXN7qEQrcEsU/cwHeuzyutibg1ItTWWtoXYV8SFnVom8NF3UwXF8I+Vj6VF1EStWCcClgPjLbiOm8vIgaouXtbUIFv25GpadXyUtwiHEq1NKUPLgka8Usm8ibyS9hY9KmNWlD9AfmnwAImc8Urn2dErcsfAltxoGUHCI7ZBHGpzg+6dpynNR+giJMrjwWzREaCHmiJX3g6jmHuuY54eSOs6cxqi3E4jNHVWXPe/tEF3l0klEvl jschmid@suse.com", "key_options": null, "keyfile": "/home/jschmid/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jschmid", "validate_certs": true} 2020-04-10T06:59:32.892 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42', u'name': u'zdover'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlnlBzRdWRqns/9VrYuF9tLILUjeGd9z8Tf/dPZmxAt3nnZqpTvI2xXemQpyt/2lsD0U24EuTc1Y/dBsgqvXtZOw51TZE8RK26hvCCaKGfcKONjvR7goHAmkYPHK9+F6m7fgYkRzA0+ZW4HcWNh/xxhhL2G1EGfz5bzIoGQJHSdzc6VeVIsY7HiYEMTxvxdsgV72Kjq97BdmHm56RBzoH24oN0UzNZS8Syx5SuWj9dA3IyFBPwjLlozSk6U2Ofd7qaXuQKl807eD9g6RhRldzHPS+i0EK3EM6Jvho5Y08k1KqV4gIEwMdzDbD3SH0xq+nUPFKcDTUFNp+tkg10GxqN zac.dover@gmail.com", "key_options": null, "keyfile": "/home/zdover/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zdover", "validate_certs": true} 2020-04-10T06:59:32.945 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'kachau@thinkpad GsZIGCdLojpvOZDxwcWSlg 0638220db65953e0cd1f134c499e4a713b32a882c8a92458afcb495dbd087518', u'name': u'kachau'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad GsZIGCdLojpvOZDxwcWSlg 0638220db65953e0cd1f134c499e4a713b32a882c8a92458afcb495dbd087518"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHZ739LVtNJVfcdnwYl7dwvPpC96dHxSn7UlRIzvSbBid0B+pMxE9OZv0UlyvDe737oPZthZYrAaeicKHO9p5aQwbhhIGImvm85sd1YxiQC6h6LxebJ6iFbtzMaEQ9SJUJFvUtItc1nQ8aPpY0CkDFWbzNn6ib8AKGTfcA+wsg08eBUJNSKexebHQCLC2c3HlDmRnjSTitl72h5pSgq7vcynhCJDlkJg49YesmNhWm9QNjwih3ITRzzUC5GaQ4uODlOETRmf84J9FDX8fMaRdPl4cZwhfI8q+0T/74Qrg1J82Ska+O5DhhbzFEqBkchYpbgetTPkUNHRbfL/GlK+7/8BKiUW+Fo7NkTDVpVYBaQOQSZEWWiPq04NMLORJ1HiHo7HsT7WI/HcVbt19YWZf7VZgB06p8LAaZsZxLBO/eqtjebOLFmMoEa4UaAepBXKo8O2hddaHjb4WVrEN+bTRUexuyTlZVbeukUky3gI2MfQirP4stmci02OKuvUuP1sM= karan@localhost.localdomain", "key_options": null, "keyfile": "/home/kachau/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kachau", "validate_certs": true} 2020-04-10T06:59:33.040 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b', u'name': u'rraja'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC4zwWf+kMEDgaSOrX7AlD+UCOgf4/fRiorQVJqC8GfMNynfiFVZaThngwIrwtQROfXHbzspi+AcRvDLewY25mXyuxVgOfSbuSIEqiUNvzcVq9IItI1J58xeDBpIktAsoHHS6JENV56fAT6krzRAQLnSa1k2b0pyKuPBzFSeNgiaQMhjrCLAaCi9yLsdKpnWgXhyfrdqvN2r/Ctts/zlTPqtRYdvdIL/p/l+vg7Q8S3OhuAGtimaEq3RE70xadaPbGhsLISMTrbfYXbxw84KKZpNYwO4zmQ2ztcvC9D7A/8ov3+A5Jq42G+pq7ORuix2NoTWtADMiwvNe1HyzS/7cX/BuXLFssRY1bzRvp+x2ExZ7b+bFx+TE/RduRh0eyDNLg7SyL83aeTr2Usa+mPoCAMKed9XtYbRnUu7ox4wmfeAgx6jM9uG5uQfauJ1XpDddrVBG8KYBvSd0BM2cyRdcAXqwxg9AVPZcdE7sbBk44CGVhw+ziZ3e3cLBTO4y44KRQre+1Bhe/Ze8OTQWLFugl/XagFlJE/L6fGUIPuFojmQvlw5dBZwtUURnxsu8yHm3iIsj4y8nR1zHMj+zuMk6rF3ZW0epPAECAdq4gG8X2wv3HTq0vLBkKpi1KMAmntEtcVwF0fzMt5XYqRd4jceFgonAAipJNDknHgi6yaUzBYmQ== rraja@redhat.com", "key_options": null, "keyfile": "/home/rraja/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rraja", "validate_certs": true} 2020-04-10T06:59:33.072 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034', u'name': u'khiremat'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "khiremat", "ovpn": "khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQujcXnq9qOCmTWs44GwKOw48BxZGBmo9AeFk3KPRx33Cz5hS7UHPYvJy03KdoMySgxJtuxsOSvjJSPxpXesTtArO5giW8RLyuVsu9q5j/GhPKyABttuGQyexiRokrHQFLVifzgqwsUARwHarWH16Oa1n6fZPFNqSH56c872zS4Pwqgkzx99NKRWh/B+fIk8VmDzzP1qvQAnUXDeTeTOzapAL+8fSNTp3QnhmZbvCHYwUxfSqJyzq1wBL+517WrvzvX7yCY9B0wzm72sIO4daV6UDyAL1B72QpB04vEZZ/skwS/0jmokfBn43HHCMPy/Mxywfi54alFJzIu4pti30V kravishankar@localhost.localdomain", "key_options": null, "keyfile": "/home/khiremat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "khiremat", "validate_certs": true} 2020-04-10T06:59:33.185 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f', u'name': u'toabctl'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2mEyxfkW1u84u5BocLCAXESj2qObWkxUd20cmPolhJYlIWLOafa69Dw5A9CjFBp0U6Jx5d0JWcwmAc5ZeffQ7t7vzmtHCZnpKkyZhN9bu6vuCh4N+I643SI/K7tdbB2E8ZE6uI7ORMAan5+t0bNSVolUVq6X1w7dIA5El54K10WNdiwgRKW6e3anNbJ3rBLGEk7gawZqDDOqTvVQmOBGWek3BjSReDvg3rkmTQLxJkiKdxnx4/s7gzH1xTpsS/OludIuSxF0BK064ZcT1t09qcjnT4Os1inV2GfYswt+UhpgqrHqXjEs0Z3y4Oy77QBKL1rDf93CP4a4+eJjqXe+v tbechtold@suse.com", "key_options": null, "keyfile": "/home/toabctl/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "toabctl", "validate_certs": true} 2020-04-10T06:59:33.218 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'kapandya@thinkpad Uw1JHj0kPIqSd9iOJ68TNg 46badc3b5cd150bfb74daf4300c8329491000521738ba70455631aa4a47e42c0', u'name': u'kapandya'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad Uw1JHj0kPIqSd9iOJ68TNg 46badc3b5cd150bfb74daf4300c8329491000521738ba70455631aa4a47e42c0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDpzfLLhjb6WlU3D0D26KAzcSgy9FkG1jnU2X6PqvyGuxyqrfDfLRc6d9A/clStS6GmyKfmevMLFr0Qn17FbPguZJ98Cnx4WwTyRyDSY6M4aoHdhKJgX6zGF7EJxCgb4gJdzlVO8uWD9k1ULV5ShS6z24rkjmxtX8Q/nkDa7w6XThLXQHPXmEJZ7ZsURaMwU+A5K0Mun/RO9W1TRKBzqspvb8D8q/o5R1T4Y6nSgMI8c3fbiuikaMogbPFTzjp154VNBD08kqTpi6JFgWUbhp6Cgpi9NJprsaPLvf1hOBnPvMENy+hkLE7TKnOqFPW6NglbB7igLOlLdqZgNfalX+XSkklZgqrJX/8/eXgGAKE4AvPDerAbBd2X9H2I5P84o5Rs8cDw2vmV5zPPT0SOJP+Ge8eXanekbVxbNI3UrrEEj/o3vQz2/ng1EpknlySpy8FIEDKF1qZ/M9Gku0RcquEZTYsUlGkxl1RYh6VhSp2W6QSeAl4B8CUedv4zQ0Cqr18= kapandya@localhost.localdomain", "key_options": null, "keyfile": "/home/kapandya/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kapandya", "validate_certs": true} 2020-04-10T06:59:33.343 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39', u'name': u'mgfritch'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCitWIN7jm5KEmB6jqcA+ufKRLodxkJ1GIdaaueCdeXdMHgWNH9FUDCUJk5dcA11MxQO15DRH/cdhb3QHPGiMv4XrBeK6HHYwGttzkcUEz1EDUhpowOrnFETbPa7K39ObmRcgdvoGXOSrNZ4mvptCTz9Xb0MF+Gcg5iJ4g+qZzIVa2/ueecYY8ZorbaH4m/NWUub6Rf01Q04HfxTvKrUFswrbCoE/Aigz/Oh88RiS854XoTvV7rE1tKbyKkaungwMAaQSNnZkq4hi8nbp6pzTxJLV7N/ZYMhVpdxSbWcghFg6cy7aeoCzZMN97WcDhpdfbhRT0dbqCkMmHV6t0usDDnOss2v68UCzkxuA1Hj4uy8rRHjXBtYqI6xsrwCjvmgP05B14TpmFdG+3GdRHx2+QlhnGj7nuPf1KoWW/f3tD85DXoEmVl6RJ4KTrMeEVlOn3kdMLVH0y2hIqPTYMP5cMVNi0EdfeoBdd9kxMKMvoSVBAgmLzGVv17o1lNvVS6V1kyt7gME2PfNR2Np+U1cJVJfUJRTm28mHXBVoDXG9s0zv8+LpQ/gVPS+vsPAfxWWYGW8ZmEV3ZorDOZycSPybjHkPFzDAdUhPG0Db0o+F9ug+22F7pvifsdDZ42zqGGLbyzfIQE3AtyI7LR7Sc5tpN9rKTeSBYJ0yNcDKEj5FUHDw== mfritch@suse.com", "key_options": null, "keyfile": "/home/mgfritch/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mgfritch", "validate_certs": true} 2020-04-10T06:59:33.365 INFO:teuthology.task.ansible.out:changed: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89', u'name': u'prasad'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNEEz49M5KdLcp0cbKrE9iihrH5PEhsPgnAxac+aQwvnTstOOlUOyR+TvHiRa7sgXHuFX/UXo+k3Qmhs5U/sMZWZmmN3Jb3fKT78lSBbGnByclfPoGVTTLOHxwo+RFE79Sp3e30oshJ3DwqYgiX6JjM8B2Y7d5Tv0Ss5uEZswvmCEjdTLDSsoxrsOjVxBpuArOfCxes8PnKDdq21mWldpP66iwFel2GgOGSc85iVHKcOE2OpVMazMzXoMSMFoOpCULKjFfXBuDIqkXwJmtLJaiCgcVmTR56KXh7JpZ7vAeJ9ttv9qgOckqKdI3jEjeDT+TrdLlRWghBE7FRkm0vmun prasad.krishnan@macprasad", "key_options": null, "keyfile": "/home/prasad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prasad", "validate_certs": true} 2020-04-10T06:59:33.476 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:33 +0000 (0:00:15.840) 0:00:43.929 ********** 2020-04-10T06:59:33.495 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'xiubli@fedora1 vosB15bA3uALkBrmjcKFUg 4ef5cb132c30563fd5fb9e4e9352786ad0d2258e41ed63f91c1f65f69b54d50d', u'name': u'xiubli'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiubli", "ovpn": "xiubli@fedora1 vosB15bA3uALkBrmjcKFUg 4ef5cb132c30563fd5fb9e4e9352786ad0d2258e41ed63f91c1f65f69b54d50d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCzVa77kg0qGGE3IpMZgSYJ+UwQiDEOg3Lji470V/sQ/R7g6/v7FZpaYqGn1wG3XeUp2gGACxNKLQfGV9EGU66NvfQYqn9A8dXRpTpZuGWN8EcKK98YrSAq0s0WzeiEzVT86RHzVHIqL6/Zn4B1p9sIeY9Hb+bfKOdtYkRBDoFELtiMRzHPFaiIJFnXl+EhH8wceYqXVNNvCuOUBMnOiDYLDtujvYdPt+ff7rvXuOzbwcAKoINFM1C4FtoUEYriKlsxJ2rr7AblfDm/WIrCZcqRBU4JcudtvZOtHhvN4uL6WgzuzuhIQFyePAeIHFUn6AiwKC/MApq7z1Qj+N3pN/Nz xiubli@fedora1", "key_options": null, "keyfile": "/home/xiubli/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiubli", "validate_certs": true} 2020-04-10T06:59:33.520 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9', u'name': u'andrewschoen'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:33.528 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n', u'name': u'zack'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:33.540 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd', u'name': u'kdreyer'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:33.550 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n', u'name': u'dmick'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:33.551 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n', u'name': u'sage'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:33.563 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n', u'name': u'dgalloway'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:33.574 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045', u'name': u'akraitma'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:33.585 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9', u'name': u'jcaratzas'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:33.596 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e', u'name': u'trociny'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:33.619 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2020-04-10T06:59:33.620 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f', u'name': u'jafaj'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDT+6B5Fr99p8xe9Zo5x8ULcdeA6ne727iVJp/xaguGwGHj6AhuK+R/CjznnHk0yA+6kiblNzjvbnsuzO8T9c8Ayv5KU07MxP1y1pymdYBpEh3NTXmhB6QOTCNTQbT14PL6MVgzK17SqarFCYXoEGuyK9jaBdN9QM76vhEaXc/2Osg+d8boJTkOk7lOPHbzeOCbQU3d+jX0VFOgAQS5OFz3Ug/X7nB0D0LN8bt5FlCZiGO95Iwls/SyPDbVXc9ot8oh4ME7kd7sElokBdYdXPauibRxWKjSDn+hV6kZfSzqwnMFaZ1TC+Av6zNsBuZucNSp3/Ws3T5i9i+21eyS4+qVE8IiX2aVKYuC9afb2lPLbdG2Iy1CqiaeGPkZfiGT9P8iQPOkqae45859UzFAd/XzKjBcmus3lSArKG1EyZl03HQ5P2J60pErCXq8W9mPjxAze2ReLRXJVEgTXnmdhCptcg6PZ4d/k4BEddbdnoX6/BEgFxLEJ64HpTeTVcDXHpipCpxFSRvgyPH/wQGKAvt8AfKBhmeZs7LuyGOhJiOon4DDaShHpE9wTnJ5qlnyg44+u6NCwelGPMAkHKEO4OgcjSsyXm+HYUpxG2s4poloIJzvU6oa3dvJsMZFhcb8CsW75sBYE+YngS/1fg1WU88ZOdaa1yF4F87pVZycpZqY1w== jan@jfsuselaptop", "key_options": null, "keyfile": "/home/jafaj/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jafaj", "validate_certs": true} 2020-04-10T06:59:33.735 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2020-04-10T06:59:33.737 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007', u'name': u'smithfarm', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "key_options": null, "keyfile": "/home/smithfarm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smithfarm", "validate_certs": true} 2020-04-10T06:59:33.747 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n', u'name': u'mbenjamin'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:33.779 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2020-04-10T06:59:33.780 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef', u'name': u'jschmid'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo8wE/b2/EDwV018ENfcENVfXSF7vnXAwBSzNVScgxdqr2DVKTWDzhoI0m5CXTqI2HLnJVFDTgN3t5/fFcmwmSAP8ic8wU1JznSPZXN7qEQrcEsU/cwHeuzyutibg1ItTWWtoXYV8SFnVom8NF3UwXF8I+Vj6VF1EStWCcClgPjLbiOm8vIgaouXtbUIFv25GpadXyUtwiHEq1NKUPLgka8Usm8ibyS9hY9KmNWlD9AfmnwAImc8Urn2dErcsfAltxoGUHCI7ZBHGpzg+6dpynNR+giJMrjwWzREaCHmiJX3g6jmHuuY54eSOs6cxqi3E4jNHVWXPe/tEF3l0klEvl jschmid@suse.com", "key_options": null, "keyfile": "/home/jschmid/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jschmid", "validate_certs": true} 2020-04-10T06:59:33.873 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01', u'name': u'aemerson', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "key_options": null, "keyfile": "/home/aemerson/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aemerson", "validate_certs": true} 2020-04-10T06:59:33.934 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2020-04-10T06:59:33.936 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'kachau@thinkpad GsZIGCdLojpvOZDxwcWSlg 0638220db65953e0cd1f134c499e4a713b32a882c8a92458afcb495dbd087518', u'name': u'kachau'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad GsZIGCdLojpvOZDxwcWSlg 0638220db65953e0cd1f134c499e4a713b32a882c8a92458afcb495dbd087518"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHZ739LVtNJVfcdnwYl7dwvPpC96dHxSn7UlRIzvSbBid0B+pMxE9OZv0UlyvDe737oPZthZYrAaeicKHO9p5aQwbhhIGImvm85sd1YxiQC6h6LxebJ6iFbtzMaEQ9SJUJFvUtItc1nQ8aPpY0CkDFWbzNn6ib8AKGTfcA+wsg08eBUJNSKexebHQCLC2c3HlDmRnjSTitl72h5pSgq7vcynhCJDlkJg49YesmNhWm9QNjwih3ITRzzUC5GaQ4uODlOETRmf84J9FDX8fMaRdPl4cZwhfI8q+0T/74Qrg1J82Ska+O5DhhbzFEqBkchYpbgetTPkUNHRbfL/GlK+7/8BKiUW+Fo7NkTDVpVYBaQOQSZEWWiPq04NMLORJ1HiHo7HsT7WI/HcVbt19YWZf7VZgB06p8LAaZsZxLBO/eqtjebOLFmMoEa4UaAepBXKo8O2hddaHjb4WVrEN+bTRUexuyTlZVbeukUky3gI2MfQirP4stmci02OKuvUuP1sM= karan@localhost.localdomain", "key_options": null, "keyfile": "/home/kachau/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kachau", "validate_certs": true} 2020-04-10T06:59:34.008 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2020-04-10T06:59:34.009 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78', u'name': u'amaredia', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "key_options": null, "keyfile": "/home/amaredia/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amaredia", "validate_certs": true} 2020-04-10T06:59:34.067 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2020-04-10T06:59:34.069 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034', u'name': u'khiremat'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "khiremat", "ovpn": "khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQujcXnq9qOCmTWs44GwKOw48BxZGBmo9AeFk3KPRx33Cz5hS7UHPYvJy03KdoMySgxJtuxsOSvjJSPxpXesTtArO5giW8RLyuVsu9q5j/GhPKyABttuGQyexiRokrHQFLVifzgqwsUARwHarWH16Oa1n6fZPFNqSH56c872zS4Pwqgkzx99NKRWh/B+fIk8VmDzzP1qvQAnUXDeTeTOzapAL+8fSNTp3QnhmZbvCHYwUxfSqJyzq1wBL+517WrvzvX7yCY9B0wzm72sIO4daV6UDyAL1B72QpB04vEZZ/skwS/0jmokfBn43HHCMPy/Mxywfi54alFJzIu4pti30V kravishankar@localhost.localdomain", "key_options": null, "keyfile": "/home/khiremat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "khiremat", "validate_certs": true} 2020-04-10T06:59:34.141 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2020-04-10T06:59:34.143 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5', u'name': u'tserlin', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "key_options": null, "keyfile": "/home/tserlin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tserlin", "validate_certs": true} 2020-04-10T06:59:34.152 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28', u'name': u'dis'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.163 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n', u'name': u'gregf'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.175 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n', u'name': u'joshd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.186 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n', u'name': u'davidz'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.198 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed', u'name': u'cmeno'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.208 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1', u'name': u'ivancich'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.219 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n', u'name': u'wusui'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.220 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2020-04-10T06:59:34.222 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'kapandya@thinkpad Uw1JHj0kPIqSd9iOJ68TNg 46badc3b5cd150bfb74daf4300c8329491000521738ba70455631aa4a47e42c0', u'name': u'kapandya'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad Uw1JHj0kPIqSd9iOJ68TNg 46badc3b5cd150bfb74daf4300c8329491000521738ba70455631aa4a47e42c0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDpzfLLhjb6WlU3D0D26KAzcSgy9FkG1jnU2X6PqvyGuxyqrfDfLRc6d9A/clStS6GmyKfmevMLFr0Qn17FbPguZJ98Cnx4WwTyRyDSY6M4aoHdhKJgX6zGF7EJxCgb4gJdzlVO8uWD9k1ULV5ShS6z24rkjmxtX8Q/nkDa7w6XThLXQHPXmEJZ7ZsURaMwU+A5K0Mun/RO9W1TRKBzqspvb8D8q/o5R1T4Y6nSgMI8c3fbiuikaMogbPFTzjp154VNBD08kqTpi6JFgWUbhp6Cgpi9NJprsaPLvf1hOBnPvMENy+hkLE7TKnOqFPW6NglbB7igLOlLdqZgNfalX+XSkklZgqrJX/8/eXgGAKE4AvPDerAbBd2X9H2I5P84o5Rs8cDw2vmV5zPPT0SOJP+Ge8eXanekbVxbNI3UrrEEj/o3vQz2/ng1EpknlySpy8FIEDKF1qZ/M9Gku0RcquEZTYsUlGkxl1RYh6VhSp2W6QSeAl4B8CUedv4zQ0Cqr18= kapandya@localhost.localdomain", "key_options": null, "keyfile": "/home/kapandya/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kapandya", "validate_certs": true} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304', u'name': u'zyan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.233 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n', u'name': u'yuriw'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.244 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n', u'name': u'tamil'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.254 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n', u'name': u'jowilkin'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.266 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4', u'name': u'bhubbard'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.267 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n', u'name': u'yehudasa'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.279 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4', u'name': u'dang'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.290 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb', u'name': u'branto'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.291 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6', u'name': u'xiaoxichen'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.302 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e', u'name': u'ffilz'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.313 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n', u'name': u'joao'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.324 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n', u'name': u'nhm'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.335 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313', u'name': u'nwatkins'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.346 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22', u'name': u'mkidd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.348 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af', u'name': u'jlopez'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.357 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb', u'name': u'haomaiwang'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.369 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608', u'name': u'jdillaman'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.380 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3', u'name': u'kchai'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.380 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2020-04-10T06:59:34.381 INFO:teuthology.task.ansible.out:changed: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89', u'name': u'prasad'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNEEz49M5KdLcp0cbKrE9iihrH5PEhsPgnAxac+aQwvnTstOOlUOyR+TvHiRa7sgXHuFX/UXo+k3Qmhs5U/sMZWZmmN3Jb3fKT78lSBbGnByclfPoGVTTLOHxwo+RFE79Sp3e30oshJ3DwqYgiX6JjM8B2Y7d5Tv0Ss5uEZswvmCEjdTLDSsoxrsOjVxBpuArOfCxes8PnKDdq21mWldpP66iwFel2GgOGSc85iVHKcOE2OpVMazMzXoMSMFoOpCULKjFfXBuDIqkXwJmtLJaiCgcVmTR56KXh7JpZ7vAeJ9ttv9qgOckqKdI3jEjeDT+TrdLlRWghBE7FRkm0vmun prasad.krishnan@macprasad", "key_options": null, "keyfile": "/home/prasad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prasad", "validate_certs": true} 2020-04-10T06:59:34.467 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2020-04-10T06:59:34.468 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n', u'name': u'dfuller'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n', u'name': u'owasserm'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.470 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n', u'name': u'vasu'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37', u'name': u'smohan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.471 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114', u'name': u'abhi'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.471 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611', u'name': u'cbodley'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.473 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195', u'name': u'mwatts'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.473 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea', u'name': u'pdonnell'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.474 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868', u'name': u'jlayton'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.513 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:34 +0000 (0:00:01.037) 0:00:44.966 ********** 2020-04-10T06:59:34.519 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f', u'name': u'rzarzynski'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.523 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e', u'name': u'rdias'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.524 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953', u'name': u'vshankar'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.526 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58', u'name': u'akupczyk'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n', u'name': u'nojha'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.527 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43', u'name': u'ifed01'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.541 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4', u'name': u'myoungwon'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.551 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9', u'name': u'andrewschoen'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.563 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n', u'name': u'zack'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.574 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd', u'name': u'kdreyer'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.574 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2', u'name': u'gabrioux'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.576 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f', u'name': u'leseb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.576 INFO:teuthology.task.ansible.out: 2020-04-10T06:59:34.578 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca', u'name': u'hchen'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n', u'name': u'dmick'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.579 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n', u'name': u'jcollin'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.589 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n', u'name': u'sage'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.590 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc', u'name': u'xxg'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.592 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n', u'name': u'dgalloway'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.602 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53', u'name': u'pcuzner'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.603 INFO:teuthology.task.ansible.out: 2020-04-10T06:59:34.604 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045', u'name': u'akraitma'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.611 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2', u'name': u'mkogan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.612 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9', u'name': u'jcaratzas'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.624 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.624 INFO:teuthology.task.ansible.out: 2020-04-10T06:59:34.626 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e', u'name': u'trociny'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736', u'name': u'henrix'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.637 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26', u'name': u'pbs1108'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.649 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315', u'name': u'clacroix'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.660 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006', u'name': u'epuertat'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.661 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461', u'name': u'tdehler'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.672 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305', u'name': u'laura'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.683 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce', u'name': u'yaarit'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.694 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3', u'name': u'rpavani1998'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.694 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4', u'name': u'rishabh'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.705 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2', u'name': u'skrah'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.730 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8', u'name': u'smanjara'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.731 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41', u'name': u'bengland'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.742 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d', u'name': u'pnawracay'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.753 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8', u'name': u'alfonsomthd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.754 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007', u'name': u'smithfarm', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "key_options": null, "keyfile": "/home/smithfarm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smithfarm", "validate_certs": true} 2020-04-10T06:59:34.755 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e', u'name': u'oliveiradan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.765 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n', u'name': u'mbenjamin'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.766 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba', u'name': u'swagner'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.777 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e', u'name': u'yuvalif'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.789 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852', u'name': u'kkeithle'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.801 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130', u'name': u'emmericp'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.802 INFO:teuthology.task.ansible.out: 2020-04-10T06:59:34.812 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0', u'name': u'mchangir'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.823 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286', u'name': u'sidharthanup'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.824 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31', u'name': u'varsha'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.835 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n', u'name': u'sjust'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.846 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'deepika@x1carbon BgvhtHMrCcelT98ekPDMoA 6fa3cf08186b93b966b68841f3507a0809d348c5097b39b6140832cac2afd511', u'name': u'ideepika'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon BgvhtHMrCcelT98ekPDMoA 6fa3cf08186b93b966b68841f3507a0809d348c5097b39b6140832cac2afd511"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.846 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46', u'name': u'gsalomon'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.857 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de', u'name': u'soumyakoduri'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.868 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a', u'name': u'kyr'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.879 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849', u'name': u'sseshasa'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.880 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189', u'name': u'rfriedma'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.891 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b', u'name': u'lmb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.902 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc', u'name': u'ksirivad'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.913 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4', u'name': u'ofriedma'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.914 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01', u'name': u'aemerson', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "key_options": null, "keyfile": "/home/aemerson/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aemerson", "validate_certs": true} 2020-04-10T06:59:34.925 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52', u'name': u'h4iresis'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.926 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5', u'name': u'shyukri'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.937 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e', u'name': u'scarvalhojr'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.948 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b', u'name': u'rakesh'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.959 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f', u'name': u'ryg-'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.963 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.975 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42', u'name': u'zdover'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.986 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b', u'name': u'rraja'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.996 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f', u'name': u'toabctl'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:34.997 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39', u'name': u'mgfritch'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.009 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'xiubli@fedora1 vosB15bA3uALkBrmjcKFUg 4ef5cb132c30563fd5fb9e4e9352786ad0d2258e41ed63f91c1f65f69b54d50d', u'name': u'xiubli'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "xiubli@fedora1 vosB15bA3uALkBrmjcKFUg 4ef5cb132c30563fd5fb9e4e9352786ad0d2258e41ed63f91c1f65f69b54d50d"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.020 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f', u'name': u'jafaj'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.031 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef', u'name': u'jschmid'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.044 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'kachau@thinkpad GsZIGCdLojpvOZDxwcWSlg 0638220db65953e0cd1f134c499e4a713b32a882c8a92458afcb495dbd087518', u'name': u'kachau'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad GsZIGCdLojpvOZDxwcWSlg 0638220db65953e0cd1f134c499e4a713b32a882c8a92458afcb495dbd087518"}, "skip_reason": "Conditional result was False"} ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78', u'name': u'amaredia', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hG 2020-04-10T06:59:35.045 INFO:teuthology.task.ansible.out:tB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "key_options": null, "keyfile": "/home/amaredia/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amaredia", "validate_certs": true} skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034', u'name': u'khiremat'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.054 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'kapandya@thinkpad Uw1JHj0kPIqSd9iOJ68TNg 46badc3b5cd150bfb74daf4300c8329491000521738ba70455631aa4a47e42c0', u'name': u'kapandya'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad Uw1JHj0kPIqSd9iOJ68TNg 46badc3b5cd150bfb74daf4300c8329491000521738ba70455631aa4a47e42c0"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.065 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'ovpn': u'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89', u'name': u'prasad'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.171 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:35 +0000 (0:00:00.657) 0:00:45.624 ********** 2020-04-10T06:59:35.191 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5', u'name': u'tserlin', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "key_options": null, "keyfile": "/home/tserlin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tserlin", "validate_certs": true} skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28', u 2020-04-10T06:59:35.193 INFO:teuthology.task.ansible.out:'name': u'dis'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.203 INFO:teuthology.task.ansible.out: TASK [users : Filter the revoked_users list] *********************************** skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n', u'name': u'gregf'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.250 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:35 +0000 (0:00:00.078) 0:00:45.703 ********** 2020-04-10T06:59:35.257 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n', u'name': u'joshd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.258 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n', u'name': u'davidz'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.260 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed', u'name': u'cmeno'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.261 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1', u'name': u'ivancich'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.262 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n', u'name': u'wusui'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.262 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304', u'name': u'zyan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.274 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n', u'name': u'yuriw'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.285 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n', u'name': u'tamil'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.296 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n', u'name': u'jowilkin'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.307 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4', u'name': u'bhubbard'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.308 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n', u'name': u'yehudasa'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.319 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4', u'name': u'dang'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.330 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb', u'name': u'branto'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.341 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6', u'name': u'xiaoxichen'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.341 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e', u'name': u'ffilz'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.352 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n', u'name': u'joao'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.363 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n', u'name': u'nhm'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.374 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313', u'name': u'nwatkins'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.386 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22', u'name': u'mkidd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.386 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af', u'name': u'jlopez'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.397 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb', u'name': u'haomaiwang'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.408 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608', u'name': u'jdillaman'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.409 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-04-10T06:59:35.411 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "name": "zcerza", "state": "absent"} 2020-04-10T06:59:35.421 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2020-04-10T06:59:35.423 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3', u'name': u'kchai'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.434 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n', u'name': u'dfuller'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.445 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n', u'name': u'owasserm'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.456 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n', u'name': u'vasu'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.467 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37', u'name': u'smohan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.468 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114', u'name': u'abhi'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.478 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611', u'name': u'cbodley'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.491 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195', u'name': u'mwatts'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.501 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea', u'name': u'pdonnell'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.512 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868', u'name': u'jlayton'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.524 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f', u'name': u'rzarzynski'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e', u'name': u'rdias'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.642 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953', u'name': u'vshankar'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "skip_reason": "Conditional result was False"} skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58', u'name': u'akupczyk'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n', u'name': u'nojha'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** ok: [smithi101.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "name": "aschoen", "state": "absent"} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21a 2020-04-10T06:59:35.645 INFO:teuthology.task.ansible.out:db3d71e09d8d5bf9d0c43', u'name': u'ifed01'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4', u'name': u'myoungwon'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "skip_reason": "Conditional result was False"} skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2', u'name': u'gabrioux'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f', u'name': u'leseb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca', u'name': u'hchen'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional res 2020-04-10T06:59:35.646 INFO:teuthology.task.ansible.out:ult was False"} skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n', u'name': u'jcollin'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc', u'name': u'xxg'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53', u'name': u'pcuzner'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2', u'name': u'mkogan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.665 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.666 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736', u'name': u'henrix'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.677 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26', u'name': u'pbs1108'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.690 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315', u'name': u'clacroix'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** ok: [smithi101.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "name": "andrew", "state": "absent"} 2020-04-10T06:59:35.690 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006', u'name': u'epuertat'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.703 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461', u'name': u'tdehler'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.723 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305', u'name': u'laura'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.725 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce', u'name': u'yaarit'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.740 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3', u'name': u'rpavani1998'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.741 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4', u'name': u'rishabh'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.752 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2', u'name': u'skrah'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.766 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8', u'name': u'smanjara'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.777 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41', u'name': u'bengland'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.777 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d', u'name': u'pnawracay'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.788 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8', u'name': u'alfonsomthd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.800 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e', u'name': u'oliveiradan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.811 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba', u'name': u'swagner'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.812 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e', u'name': u'yuvalif'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.823 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852', u'name': u'kkeithle'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.834 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130', u'name': u'emmericp'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.835 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-04-10T06:59:35.835 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "name": "sweil", "state": "absent"} 2020-04-10T06:59:35.846 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2020-04-10T06:59:35.847 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0', u'name': u'mchangir'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.857 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286', u'name': u'sidharthanup'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.858 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31', u'name': u'varsha'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.869 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n', u'name': u'sjust'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.880 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'deepika@x1carbon BgvhtHMrCcelT98ekPDMoA 6fa3cf08186b93b966b68841f3507a0809d348c5097b39b6140832cac2afd511', u'name': u'ideepika'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon BgvhtHMrCcelT98ekPDMoA 6fa3cf08186b93b966b68841f3507a0809d348c5097b39b6140832cac2afd511"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.891 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46', u'name': u'gsalomon'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.901 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de', u'name': u'soumyakoduri'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.903 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a', u'name': u'kyr'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.922 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849', u'name': u'sseshasa'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.922 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189', u'name': u'rfriedma'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.924 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b', u'name': u'lmb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.944 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc', u'name': u'ksirivad'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.945 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4', u'name': u'ofriedma'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.956 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52', u'name': u'h4iresis'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.957 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5', u'name': u'shyukri'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.969 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e', u'name': u'scarvalhojr'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.979 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-04-10T06:59:35.980 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "name": "brad", "state": "absent"} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b', u'name': u'rakesh'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:35.993 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f', u'name': u'ryg-'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:36.003 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42', u'name': u'zdover'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:36.017 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b', u'name': u'rraja'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:36.030 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f', u'name': u'toabctl'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:36.041 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39', u'name': u'mgfritch'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:36.042 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'xiubli@fedora1 vosB15bA3uALkBrmjcKFUg 4ef5cb132c30563fd5fb9e4e9352786ad0d2258e41ed63f91c1f65f69b54d50d', u'name': u'xiubli'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "xiubli@fedora1 vosB15bA3uALkBrmjcKFUg 4ef5cb132c30563fd5fb9e4e9352786ad0d2258e41ed63f91c1f65f69b54d50d"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:36.053 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f', u'name': u'jafaj'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:36.064 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef', u'name': u'jschmid'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:36.075 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'kachau@thinkpad GsZIGCdLojpvOZDxwcWSlg 0638220db65953e0cd1f134c499e4a713b32a882c8a92458afcb495dbd087518', u'name': u'kachau'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad GsZIGCdLojpvOZDxwcWSlg 0638220db65953e0cd1f134c499e4a713b32a882c8a92458afcb495dbd087518"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:36.086 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034', u'name': u'khiremat'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:36.100 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'kapandya@thinkpad Uw1JHj0kPIqSd9iOJ68TNg 46badc3b5cd150bfb74daf4300c8329491000521738ba70455631aa4a47e42c0', u'name': u'kapandya'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad Uw1JHj0kPIqSd9iOJ68TNg 46badc3b5cd150bfb74daf4300c8329491000521738ba70455631aa4a47e42c0"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:36.111 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'ovpn': u'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89', u'name': u'prasad'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:36.132 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-04-10T06:59:36.133 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "name": "kefu", "state": "absent"} 2020-04-10T06:59:36.204 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:36 +0000 (0:00:00.954) 0:00:46.657 ********** 2020-04-10T06:59:36.242 INFO:teuthology.task.ansible.out: TASK [users : Filter the revoked_users list] *********************************** 2020-04-10T06:59:36.244 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** 2020-04-10T06:59:36.245 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "name": "shylesh", "state": "absent"} 2020-04-10T06:59:36.290 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:36 +0000 (0:00:00.085) 0:00:46.743 ********** 2020-04-10T06:59:36.457 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "name": "zcerza", "state": "absent"} 2020-04-10T06:59:36.576 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "name": "aschoen", "state": "absent"} 2020-04-10T06:59:36.703 INFO:teuthology.task.ansible.out:changed: [smithi101.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "gmeno", "name": "gmeno", "remove": false, "state": "absent"} 2020-04-10T06:59:36.705 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "name": "andrew", "state": "absent"} 2020-04-10T06:59:36.859 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "name": "trhoden", "state": "absent"} 2020-04-10T06:59:36.861 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "name": "sweil", "state": "absent"} 2020-04-10T06:59:37.008 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "name": "brad", "state": "absent"} 2020-04-10T06:59:37.010 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "name": "nishtha", "state": "absent"} 2020-04-10T06:59:37.147 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "name": "kefu", "state": "absent"} 2020-04-10T06:59:37.159 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "name": "yguang", "state": "absent"} 2020-04-10T06:59:37.296 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "name": "sdieffen", "state": "absent"} 2020-04-10T06:59:37.318 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "name": "shylesh", "state": "absent"} 2020-04-10T06:59:37.434 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "name": "brian", "state": "absent"} 2020-04-10T06:59:37.561 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "name": "pmcgarry", "state": "absent"} 2020-04-10T06:59:37.691 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "name": "karnan", "state": "absent"} 2020-04-10T06:59:37.725 INFO:teuthology.task.ansible.out:changed: [smithi204.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "gmeno", "name": "gmeno", "remove": false, "state": "absent"} 2020-04-10T06:59:37.830 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "name": "ryneli", "state": "absent"} 2020-04-10T06:59:37.851 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "name": "trhoden", "state": "absent"} 2020-04-10T06:59:37.963 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "name": "dlambrig", "state": "absent"} 2020-04-10T06:59:37.995 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "name": "nishtha", "state": "absent"} 2020-04-10T06:59:38.105 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "name": "icolle", "state": "absent"} 2020-04-10T06:59:38.126 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "name": "yguang", "state": "absent"} 2020-04-10T06:59:38.222 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "name": "soumya", "state": "absent"} 2020-04-10T06:59:38.264 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "name": "sdieffen", "state": "absent"} 2020-04-10T06:59:38.355 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "name": "jspray", "state": "absent"} 2020-04-10T06:59:38.388 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "name": "brian", "state": "absent"} 2020-04-10T06:59:38.491 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "name": "loic", "state": "absent"} 2020-04-10T06:59:38.537 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "name": "pmcgarry", "state": "absent"} 2020-04-10T06:59:38.621 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "name": "erwan", "state": "absent"} 2020-04-10T06:59:38.688 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "name": "karnan", "state": "absent"} 2020-04-10T06:59:38.763 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "name": "jj", "state": "absent"} 2020-04-10T06:59:38.827 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "name": "ryneli", "state": "absent"} 2020-04-10T06:59:38.901 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "name": "amarangone", "state": "absent"} 2020-04-10T06:59:38.955 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "name": "dlambrig", "state": "absent"} 2020-04-10T06:59:39.044 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "name": "oprypin", "state": "absent"} 2020-04-10T06:59:39.099 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "name": "icolle", "state": "absent"} 2020-04-10T06:59:39.190 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "name": "adamyanova", "state": "absent"} 2020-04-10T06:59:39.260 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "name": "soumya", "state": "absent"} 2020-04-10T06:59:39.326 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "name": "sbillah", "state": "absent"} 2020-04-10T06:59:39.399 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "name": "jspray", "state": "absent"} 2020-04-10T06:59:39.451 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "name": "onyb", "state": "absent"} 2020-04-10T06:59:39.548 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "name": "loic", "state": "absent"} 2020-04-10T06:59:39.580 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "name": "jwilliamson", "state": "absent"} 2020-04-10T06:59:39.688 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "name": "erwan", "state": "absent"} 2020-04-10T06:59:39.710 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "name": "kmroz", "state": "absent"} 2020-04-10T06:59:39.831 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "name": "jj", "state": "absent"} 2020-04-10T06:59:39.843 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "name": "shehbazj", "state": "absent"} 2020-04-10T06:59:39.955 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "name": "amarangone", "state": "absent"} 2020-04-10T06:59:39.987 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "name": "abhishekvrshny", "state": "absent"} 2020-04-10T06:59:40.072 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "name": "oprypin", "state": "absent"} 2020-04-10T06:59:40.134 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "name": "asheplyakov", "state": "absent"} 2020-04-10T06:59:40.211 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "name": "adamyanova", "state": "absent"} 2020-04-10T06:59:40.255 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "name": "liupan", "state": "absent"} 2020-04-10T06:59:40.349 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "name": "sbillah", "state": "absent"} 2020-04-10T06:59:40.383 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "name": "adeza", "state": "absent"} 2020-04-10T06:59:40.469 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "name": "onyb", "state": "absent"} 2020-04-10T06:59:40.512 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "name": "nlevine", "state": "absent"} 2020-04-10T06:59:40.598 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "name": "jwilliamson", "state": "absent"} 2020-04-10T06:59:40.672 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "name": "tbrekke", "state": "absent"} 2020-04-10T06:59:40.737 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "name": "kmroz", "state": "absent"} 2020-04-10T06:59:40.811 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "name": "taco", "state": "absent"} 2020-04-10T06:59:40.868 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "name": "shehbazj", "state": "absent"} 2020-04-10T06:59:40.942 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "name": "louis", "state": "absent"} 2020-04-10T06:59:41.006 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "name": "abhishekvrshny", "state": "absent"} 2020-04-10T06:59:41.079 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "name": "amarango", "state": "absent"} 2020-04-10T06:59:41.137 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "name": "asheplyakov", "state": "absent"} 2020-04-10T06:59:41.212 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "name": "oobe", "state": "absent"} 2020-04-10T06:59:41.282 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "name": "liupan", "state": "absent"} 2020-04-10T06:59:41.365 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "name": "rturk", "state": "absent"} 2020-04-10T06:59:41.418 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "name": "adeza", "state": "absent"} 2020-04-10T06:59:41.499 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "name": "fche", "state": "absent"} 2020-04-10T06:59:41.571 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:41 +0000 (0:00:05.281) 0:00:52.024 ********** 2020-04-10T06:59:41.579 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "name": "nlevine", "state": "absent"} 2020-04-10T06:59:41.698 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "name": "tbrekke", "state": "absent"} 2020-04-10T06:59:41.835 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "name": "taco", "state": "absent"} 2020-04-10T06:59:41.963 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2020-04-10T06:59:42.143 INFO:teuthology.task.ansible.out:changed: [smithi101.front.sepia.ceph.com] => {"changed": true, "checksum": "196f1d5dd771b87e137fd95ccd740950ae92c1e3", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "md5sum": "c78e7333da9f7cc65be7398e99e22a31", "mode": "0644", "owner": "root", "size": 40, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1586501981.59-33279777097682/source", "state": "file", "uid": 0} Friday 10 April 2020 06:59:42 +0000 (0:00:00.438) 0:00:52.462 ********** TASK [users : Remove revoked users] ******************************************** ok: [smithi204.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "name": "louis", "state": "absent"} TASK [common : Log the OS name, version and release] *************************** ok: [smithi101.front.sepia.ceph.com] => { "msg": "Host smithi101.front.sepia.ceph.com is running Ubuntu 18.04 (bionic)" } Friday 10 April 2020 06:59:42 +0000 (0:00:00.074) 0:00:52.537 ********** TASK [common : Including major version specific variables.] ******************** ok: [smithi101.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} 2020-04-10T06:59:42.170 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:42 +0000 (0:00:00.086) 0:00:52.623 ********** 2020-04-10T06:59:42.179 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** ok: [smithi204.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "name": "amarango", "state": "absent"} 2020-04-10T06:59:42.188 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** 2020-04-10T06:59:42.190 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:42.237 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:42 +0000 (0:00:00.066) 0:00:52.690 ********** 2020-04-10T06:59:42.264 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** 2020-04-10T06:59:42.265 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:42.314 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:42 +0000 (0:00:00.075) 0:00:52.766 ********** 2020-04-10T06:59:42.319 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-04-10T06:59:42.320 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "name": "oobe", "state": "absent"} 2020-04-10T06:59:42.343 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:42.385 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:42 +0000 (0:00:00.072) 0:00:52.838 ********** 2020-04-10T06:59:42.414 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:42.459 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:42 +0000 (0:00:00.073) 0:00:52.912 ********** 2020-04-10T06:59:42.465 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-04-10T06:59:42.466 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "name": "rturk", "state": "absent"} 2020-04-10T06:59:42.478 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:42.517 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:42 +0000 (0:00:00.058) 0:00:52.970 ********** 2020-04-10T06:59:42.543 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** 2020-04-10T06:59:42.545 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:42.588 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:42 +0000 (0:00:00.070) 0:00:53.041 ********** 2020-04-10T06:59:42.594 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-04-10T06:59:42.595 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "name": "fche", "state": "absent"} 2020-04-10T06:59:42.618 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** 2020-04-10T06:59:42.620 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:42.660 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:42 +0000 (0:00:00.071) 0:00:53.113 ********** 2020-04-10T06:59:42.696 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:42 +0000 (0:00:00.036) 0:00:53.149 ********** 2020-04-10T06:59:42.702 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** 2020-04-10T06:59:42.703 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:42.745 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:42 +0000 (0:00:00.048) 0:00:53.198 ********** 2020-04-10T06:59:42.781 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** 2020-04-10T06:59:42.782 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [smithi101.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-04-10T06:59:42.823 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:42 +0000 (0:00:00.078) 0:00:53.276 ********** 2020-04-10T06:59:42.852 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** 2020-04-10T06:59:42.853 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:42.903 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:42 +0000 (0:00:00.080) 0:00:53.357 ********** 2020-04-10T06:59:42.929 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** 2020-04-10T06:59:42.931 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:42.971 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:42 +0000 (0:00:00.067) 0:00:53.424 ********** 2020-04-10T06:59:42.998 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** 2020-04-10T06:59:43.009 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-04-10T06:59:43.047 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:43 +0000 (0:00:00.076) 0:00:53.501 ********** 2020-04-10T06:59:43.074 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** 2020-04-10T06:59:43.075 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:43.118 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:43 +0000 (0:00:00.070) 0:00:53.571 ********** 2020-04-10T06:59:43.125 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2020-04-10T06:59:43.125 INFO:teuthology.task.ansible.out:changed: [smithi204.front.sepia.ceph.com] => {"changed": true, "checksum": "196f1d5dd771b87e137fd95ccd740950ae92c1e3", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "md5sum": "c78e7333da9f7cc65be7398e99e22a31", "mode": "0644", "owner": "root", "size": 40, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1586501982.72-273060428578144/source", "state": "file", "uid": 0} 2020-04-10T06:59:43.169 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:43 +0000 (0:00:00.050) 0:00:53.622 ********** 2020-04-10T06:59:43.174 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** 2020-04-10T06:59:43.175 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-04-10T06:59:43.217 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:43 +0000 (0:00:00.048) 0:00:53.670 ********** 2020-04-10T06:59:43.223 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** 2020-04-10T06:59:43.223 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => { "msg": "Host smithi204.front.sepia.ceph.com is running Ubuntu 18.04 (bionic)" } 2020-04-10T06:59:43.268 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:43 +0000 (0:00:00.050) 0:00:53.721 ********** 2020-04-10T06:59:43.273 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** 2020-04-10T06:59:43.274 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:43.316 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:43 +0000 (0:00:00.048) 0:00:53.769 ********** 2020-04-10T06:59:43.322 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** 2020-04-10T06:59:43.324 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} 2020-04-10T06:59:43.363 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:43 +0000 (0:00:00.046) 0:00:53.816 ********** 2020-04-10T06:59:43.368 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** 2020-04-10T06:59:43.369 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:43.411 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:43 +0000 (0:00:00.047) 0:00:53.864 ********** 2020-04-10T06:59:43.417 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** 2020-04-10T06:59:43.418 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:43.465 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:43 +0000 (0:00:00.049) 0:00:53.914 ********** 2020-04-10T06:59:43.467 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:43.522 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:43 +0000 (0:00:00.061) 0:00:53.975 ********** 2020-04-10T06:59:43.527 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** 2020-04-10T06:59:43.529 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:43.569 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:43 +0000 (0:00:00.046) 0:00:54.022 ********** 2020-04-10T06:59:43.574 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** 2020-04-10T06:59:43.574 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:43.614 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:43 +0000 (0:00:00.045) 0:00:54.067 ********** 2020-04-10T06:59:43.619 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* 2020-04-10T06:59:43.621 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:43.659 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:43 +0000 (0:00:00.045) 0:00:54.113 ********** 2020-04-10T06:59:43.664 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* 2020-04-10T06:59:43.666 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-04-10T06:59:43.712 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:43 +0000 (0:00:00.052) 0:00:54.165 ********** 2020-04-10T06:59:43.718 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** 2020-04-10T06:59:43.719 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:43.764 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:43 +0000 (0:00:00.051) 0:00:54.217 ********** 2020-04-10T06:59:43.765 INFO:teuthology.task.ansible.out: 2020-04-10T06:59:43.770 INFO:teuthology.task.ansible.out: 2020-04-10T06:59:43.772 INFO:teuthology.task.ansible.out:TASK [common : Clean yum cache] ************************************************ skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:43.812 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:43 +0000 (0:00:00.048) 0:00:54.265 ********** 2020-04-10T06:59:43.814 INFO:teuthology.task.ansible.out: 2020-04-10T06:59:43.819 INFO:teuthology.task.ansible.out: 2020-04-10T06:59:43.821 INFO:teuthology.task.ansible.out:TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:43.864 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:43 +0000 (0:00:00.051) 0:00:54.317 ********** 2020-04-10T06:59:43.871 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* 2020-04-10T06:59:43.872 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:43.915 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:43 +0000 (0:00:00.051) 0:00:54.368 ********** 2020-04-10T06:59:43.921 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** 2020-04-10T06:59:43.923 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:43.963 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:43 +0000 (0:00:00.048) 0:00:54.417 ********** 2020-04-10T06:59:43.973 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:44.013 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:44 +0000 (0:00:00.049) 0:00:54.466 ********** 2020-04-10T06:59:44.018 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** 2020-04-10T06:59:44.020 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:44.060 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:44 +0000 (0:00:00.047) 0:00:54.513 ********** 2020-04-10T06:59:44.065 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** 2020-04-10T06:59:44.067 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:44.112 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:44 +0000 (0:00:00.051) 0:00:54.565 ********** 2020-04-10T06:59:44.118 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** 2020-04-10T06:59:44.119 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:44.162 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:44 +0000 (0:00:00.047) 0:00:54.612 ********** 2020-04-10T06:59:44.164 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** 2020-04-10T06:59:44.165 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:44.207 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:44 +0000 (0:00:00.047) 0:00:54.660 ********** 2020-04-10T06:59:44.213 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** 2020-04-10T06:59:44.213 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-04-10T06:59:44.215 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-04-10T06:59:44.254 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:44 +0000 (0:00:00.047) 0:00:54.707 ********** 2020-04-10T06:59:44.260 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:44.306 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:44 +0000 (0:00:00.051) 0:00:54.759 ********** 2020-04-10T06:59:44.312 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** 2020-04-10T06:59:44.313 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:44.353 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:44 +0000 (0:00:00.046) 0:00:54.806 ********** 2020-04-10T06:59:44.359 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** skipping: [smithi101.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-04-10T06:59:44.413 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:44 +0000 (0:00:00.048) 0:00:54.854 ********** TASK [common : Install CA Cert from Satellite Server] ************************** skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:44.447 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:44 +0000 (0:00:00.046) 0:00:54.900 ********** 2020-04-10T06:59:44.453 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ 2020-04-10T06:59:44.500 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:44 +0000 (0:00:00.052) 0:00:54.953 ********** 2020-04-10T06:59:44.507 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** 2020-04-10T06:59:44.507 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-04-10T06:59:44.548 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:44 +0000 (0:00:00.047) 0:00:55.001 ********** 2020-04-10T06:59:44.553 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* 2020-04-10T06:59:44.554 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:44.595 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:44 +0000 (0:00:00.047) 0:00:55.048 ********** 2020-04-10T06:59:44.601 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** 2020-04-10T06:59:44.602 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:44.642 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:44 +0000 (0:00:00.047) 0:00:55.096 ********** 2020-04-10T06:59:44.647 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** 2020-04-10T06:59:44.648 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:44.691 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:44 +0000 (0:00:00.048) 0:00:55.144 ********** 2020-04-10T06:59:44.697 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** 2020-04-10T06:59:44.699 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-04-10T06:59:44.738 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:44 +0000 (0:00:00.047) 0:00:55.191 ********** 2020-04-10T06:59:44.744 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2020-04-10T06:59:44.746 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={'value': {u'gpgcheck': 0, u'mirrorlist': u'file:///etc/yum.repos.d/epel-mirrorlist', u'enabled': u'1', u'name': u'Extra Packages for Enterprise Linux', u'failovermethod': u'priority'}, 'key': u'epel'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:44.747 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={'value': {u'gpgcheck': 0, u'mirrorlist': u'file:///etc/yum.repos.d/epel-testing-mirrorlist', u'enabled': 0, u'name': u'Extra Packages for Enterprise Linux - Testing', u'failovermethod': u'priority'}, 'key': u'epel-testing'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:44.793 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:44 +0000 (0:00:00.054) 0:00:55.246 ********** 2020-04-10T06:59:44.799 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** 2020-04-10T06:59:44.800 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:44.848 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:44 +0000 (0:00:00.055) 0:00:55.301 ********** 2020-04-10T06:59:44.854 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* skipping: [smithi101.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-mirrorlist", "skip_reason": "Conditional result was False"} 2020-04-10T06:59:44.855 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-testing-mirrorlist", "skip_reason": "Conditional result was False"} 2020-04-10T06:59:44.902 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:44 +0000 (0:00:00.054) 0:00:55.355 ********** 2020-04-10T06:59:44.908 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** 2020-04-10T06:59:44.910 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:44.950 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:44 +0000 (0:00:00.047) 0:00:55.403 ********** 2020-04-10T06:59:44.958 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:45.001 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:44 +0000 (0:00:00.051) 0:00:55.454 ********** 2020-04-10T06:59:45.008 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** 2020-04-10T06:59:45.009 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:45.049 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:45 +0000 (0:00:00.047) 0:00:55.502 ********** 2020-04-10T06:59:45.087 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:45.117 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:45 +0000 (0:00:00.067) 0:00:55.570 ********** 2020-04-10T06:59:45.155 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* 2020-04-10T06:59:45.156 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Get the current timezone] *************************************** ok: [smithi101.front.sepia.ceph.com] => {"changed": false, "cmd": "cat /etc/timezone", "delta": "0:00:00.021294", "end": "2020-04-10 06:59:45.134963", "rc": 0, "start": "2020-04-10 06:59:45.113669", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2020-04-10T06:59:45.217 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:45 +0000 (0:00:00.089) 0:00:55.659 ********** 2020-04-10T06:59:45.245 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:45 +0000 (0:00:00.039) 0:00:55.698 ********** 2020-04-10T06:59:45.253 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:45.296 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:45 +0000 (0:00:00.050) 0:00:55.749 ********** 2020-04-10T06:59:45.303 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2020-04-10T06:59:45.304 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:45.345 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:45 +0000 (0:00:00.049) 0:00:55.798 ********** 2020-04-10T06:59:45.354 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:45.398 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:45 +0000 (0:00:00.052) 0:00:55.851 ********** 2020-04-10T06:59:45.409 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:45.447 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:45 +0000 (0:00:00.049) 0:00:55.900 ********** 2020-04-10T06:59:45.452 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2020-04-10T06:59:45.452 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:45.500 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:45 +0000 (0:00:00.053) 0:00:55.953 ********** 2020-04-10T06:59:45.506 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:45.560 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:45 +0000 (0:00:00.059) 0:00:56.013 ********** 2020-04-10T06:59:45.565 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** 2020-04-10T06:59:45.567 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:45.609 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:45 +0000 (0:00:00.048) 0:00:56.062 ********** 2020-04-10T06:59:45.621 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:45.654 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:45 +0000 (0:00:00.045) 0:00:56.108 ********** 2020-04-10T06:59:45.659 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ 2020-04-10T06:59:45.661 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:45.700 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:45 +0000 (0:00:00.045) 0:00:56.153 ********** 2020-04-10T06:59:45.706 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:45.746 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:45 +0000 (0:00:00.045) 0:00:56.199 ********** 2020-04-10T06:59:45.751 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* 2020-04-10T06:59:45.752 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:45.798 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:45 +0000 (0:00:00.052) 0:00:56.251 ********** 2020-04-10T06:59:45.804 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* 2020-04-10T06:59:45.805 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:45.844 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:45 +0000 (0:00:00.045) 0:00:56.297 ********** 2020-04-10T06:59:45.849 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** 2020-04-10T06:59:45.850 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:45.889 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:45 +0000 (0:00:00.045) 0:00:56.342 ********** 2020-04-10T06:59:45.895 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** 2020-04-10T06:59:45.896 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-04-10T06:59:45.936 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:45 +0000 (0:00:00.046) 0:00:56.389 ********** 2020-04-10T06:59:45.962 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ 2020-04-10T06:59:46.004 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:45 +0000 (0:00:00.068) 0:00:56.457 ********** 2020-04-10T06:59:46.042 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* 2020-04-10T06:59:46.043 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:46.082 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:46 +0000 (0:00:00.077) 0:00:56.535 ********** 2020-04-10T06:59:46.120 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:46.157 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:46 +0000 (0:00:00.074) 0:00:56.610 ********** 2020-04-10T06:59:46.195 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** skipping: [smithi204.front.sepia.ceph.com] => (item={'value': {u'gpgcheck': 0, u'mirrorlist': u'file:///etc/yum.repos.d/epel-mirrorlist', u'enabled': u'1', u'name': u'Extra Packages for Enterprise Linux', u'failovermethod': u'priority'}, 'key': u'epel'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:46.196 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={'value': {u'gpgcheck': 0, u'mirrorlist': u'file:///etc/yum.repos.d/epel-testing-mirrorlist', u'enabled': 0, u'name': u'Extra Packages for Enterprise Linux - Testing', u'failovermethod': u'priority'}, 'key': u'epel-testing'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:46.241 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:46 +0000 (0:00:00.084) 0:00:56.694 ********** 2020-04-10T06:59:46.285 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* skipping: [smithi204.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-mirrorlist", "skip_reason": "Conditional result was False"} skipping: [smithi204.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-testing-mirrorlist", "skip_reason": "Conditional result was False"} 2020-04-10T06:59:46.323 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:46 +0000 (0:00:00.081) 0:00:56.776 ********** 2020-04-10T06:59:46.349 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2020-04-10T06:59:46.351 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:46.391 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:46 +0000 (0:00:00.068) 0:00:56.845 ********** 2020-04-10T06:59:46.523 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** 2020-04-10T06:59:46.523 INFO:teuthology.task.ansible.out:changed: [smithi101.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.500634", "end": "2020-04-10 06:59:46.500369", "rc": 0, "start": "2020-04-10 06:59:45.999735", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-04-10T06:59:46.574 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:46 +0000 (0:00:00.182) 0:00:57.028 ********** 2020-04-10T06:59:46.601 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** 2020-04-10T06:59:46.602 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:46.641 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:46 +0000 (0:00:00.066) 0:00:57.094 ********** 2020-04-10T06:59:46.802 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2020-04-10T06:59:46.803 INFO:teuthology.task.ansible.out: [WARNING]: Consider using the apt module rather than running 'apt-get'. If you need to use command because apt is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message. 2020-04-10T06:59:46.803 INFO:teuthology.task.ansible.out:changed: [smithi101.front.sepia.ceph.com] => {"changed": true, "cmd": ["apt-get", "clean"], "delta": "0:00:00.043403", "end": "2020-04-10 06:59:46.786111", "rc": 0, "start": "2020-04-10 06:59:46.742708", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-04-10T06:59:46.849 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:46 +0000 (0:00:00.208) 0:00:57.302 ********** 2020-04-10T06:59:46.858 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2020-04-10T06:59:46.859 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => {"changed": false, "cmd": "cat /etc/timezone", "delta": "0:00:00.266614", "end": "2020-04-10 06:59:47.021124", "rc": 0, "start": "2020-04-10 06:59:46.754510", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2020-04-10T06:59:46.918 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:46 +0000 (0:00:00.068) 0:00:57.371 ********** 2020-04-10T06:59:46.946 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** 2020-04-10T06:59:46.947 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:46.985 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:46 +0000 (0:00:00.067) 0:00:57.438 ********** 2020-04-10T06:59:47.017 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:47.064 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:47 +0000 (0:00:00.078) 0:00:57.517 ********** 2020-04-10T06:59:47.092 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:47.132 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:47 +0000 (0:00:00.066) 0:00:57.584 ********** 2020-04-10T06:59:47.157 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:47.196 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:47 +0000 (0:00:00.065) 0:00:57.649 ********** 2020-04-10T06:59:47.221 INFO:teuthology.task.ansible.out: 2020-04-10T06:59:47.223 INFO:teuthology.task.ansible.out:TASK [common : Add base OpenSUSE Leap repo] ************************************ skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:47.261 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:47 +0000 (0:00:00.065) 0:00:57.714 ********** 2020-04-10T06:59:47.288 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:47.325 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:47 +0000 (0:00:00.063) 0:00:57.778 ********** 2020-04-10T06:59:47.361 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** 2020-04-10T06:59:47.363 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:47.404 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:47 +0000 (0:00:00.079) 0:00:57.857 ********** 2020-04-10T06:59:48.289 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** 2020-04-10T06:59:48.291 INFO:teuthology.task.ansible.out:changed: [smithi204.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.724584", "end": "2020-04-10 06:59:48.500373", "rc": 0, "start": "2020-04-10 06:59:47.775789", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-04-10T06:59:48.342 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:48 +0000 (0:00:00.929) 0:00:58.786 ********** 2020-04-10T06:59:48.365 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** 2020-04-10T06:59:48.366 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:48.405 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:48 +0000 (0:00:00.071) 0:00:58.858 ********** 2020-04-10T06:59:48.577 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2020-04-10T06:59:48.579 INFO:teuthology.task.ansible.out:changed: [smithi204.front.sepia.ceph.com] => {"changed": true, "cmd": ["apt-get", "clean"], "delta": "0:00:00.041468", "end": "2020-04-10 06:59:48.786878", "rc": 0, "start": "2020-04-10 06:59:48.745410", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-04-10T06:59:48.616 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:48 +0000 (0:00:00.211) 0:00:59.069 ********** 2020-04-10T06:59:50.561 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** [WARNING]: Could not find aptitude. Using apt-get instead ok: [smithi101.front.sepia.ceph.com] => {"attempts": 1, "cache_update_time": 1586501716, "cache_updated": false, "changed": false} 2020-04-10T06:59:50.610 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:50 +0000 (0:00:01.993) 0:01:01.063 ********** 2020-04-10T06:59:51.487 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** ok: [smithi101.front.sepia.ceph.com] => {"cache_update_time": 1586501716, "cache_updated": false, "changed": false} 2020-04-10T06:59:51.528 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:51 +0000 (0:00:00.918) 0:01:01.981 ********** 2020-04-10T06:59:51.554 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2020-04-10T06:59:51.556 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:51.594 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:51 +0000 (0:00:00.065) 0:01:02.047 ********** 2020-04-10T06:59:52.126 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** 2020-04-10T06:59:52.127 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => {"attempts": 1, "cache_update_time": 1586501758, "cache_updated": false, "changed": false} 2020-04-10T06:59:52.177 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:52 +0000 (0:00:00.582) 0:01:02.630 ********** 2020-04-10T06:59:52.261 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* ok: [smithi101.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "size": 131, "state": "file", "uid": 0} 2020-04-10T06:59:52.317 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:52 +0000 (0:00:00.139) 0:01:02.770 ********** 2020-04-10T06:59:53.052 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2020-04-10T06:59:53.053 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => {"cache_update_time": 1586501758, "cache_updated": false, "changed": false} 2020-04-10T06:59:53.092 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:53 +0000 (0:00:00.775) 0:01:03.545 ********** 2020-04-10T06:59:53.118 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2020-04-10T06:59:53.120 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:53.158 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:53 +0000 (0:00:00.065) 0:01:03.611 ********** 2020-04-10T06:59:53.173 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** 2020-04-10T06:59:53.175 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => {"cache_update_time": 1586501716, "cache_updated": false, "changed": false} 2020-04-10T06:59:53.231 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:53 +0000 (0:00:00.072) 0:01:03.684 ********** 2020-04-10T06:59:53.500 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* 2020-04-10T06:59:53.501 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "size": 131, "state": "file", "uid": 0} 2020-04-10T06:59:53.553 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:53 +0000 (0:00:00.322) 0:01:04.006 ********** 2020-04-10T06:59:53.583 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2020-04-10T06:59:53.584 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "size": 2716736, "state": "file", "uid": 0} 2020-04-10T06:59:53.877 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "size": 1952038, "state": "file", "uid": 0} 2020-04-10T06:59:54.160 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "size": 580432, "state": "file", "uid": 0} 2020-04-10T06:59:54.200 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:54 +0000 (0:00:00.646) 0:01:04.653 ********** 2020-04-10T06:59:54.320 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* 2020-04-10T06:59:54.321 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "size": 4096, "state": "directory", "uid": 0} 2020-04-10T06:59:54.365 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:54 +0000 (0:00:00.165) 0:01:04.818 ********** 2020-04-10T06:59:54.420 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** 2020-04-10T06:59:54.421 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => {"cache_update_time": 1586501758, "cache_updated": false, "changed": false} 2020-04-10T06:59:54.471 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:54 +0000 (0:00:00.105) 0:01:04.924 ********** 2020-04-10T06:59:54.676 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-04-10T06:59:54.683 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "9a447e184389aadb9aaf1e77f30f0a7f8b716c4e", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "size": 12725, "state": "file", "uid": 0} 2020-04-10T06:59:54.893 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** ok: [smithi204.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "size": 2716736, "state": "file", "uid": 0} 2020-04-10T06:59:54.935 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-04-10T06:59:54.936 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "size": 8602, "state": "file", "uid": 0} 2020-04-10T06:59:55.218 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2020-04-10T06:59:55.219 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "size": 1952038, "state": "file", "uid": 0} 2020-04-10T06:59:55.222 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-04-10T06:59:55.224 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "size": 3605, "state": "file", "uid": 0} 2020-04-10T06:59:55.272 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:55 +0000 (0:00:00.801) 0:01:05.725 ********** 2020-04-10T06:59:55.309 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2020-04-10T06:59:55.311 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib/nagios/plugins", "nrpe_group": "nagios", "nrpe_packages": ["nagios-nrpe-server", "nagios-plugins-basic"], "nrpe_service_name": "nagios-nrpe-server", "nrpe_user": "nagios"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/apt_systems.yml"], "changed": false} 2020-04-10T06:59:55.357 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:55 +0000 (0:00:00.084) 0:01:05.810 ********** 2020-04-10T06:59:55.394 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2020-04-10T06:59:55.395 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:55.437 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:55 +0000 (0:00:00.079) 0:01:05.890 ********** 2020-04-10T06:59:55.463 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* 2020-04-10T06:59:55.465 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-04-10T06:59:55.504 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:55 +0000 (0:00:00.066) 0:01:05.957 ********** 2020-04-10T06:59:55.521 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2020-04-10T06:59:55.523 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "size": 580432, "state": "file", "uid": 0} 2020-04-10T06:59:55.585 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:55 +0000 (0:00:00.080) 0:01:06.038 ********** 2020-04-10T06:59:55.589 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2020-04-10T06:59:55.590 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-04-10T06:59:55.591 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-04-10T06:59:55.592 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-04-10T06:59:55.593 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-04-10T06:59:55.635 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:55 +0000 (0:00:00.050) 0:01:06.088 ********** 2020-04-10T06:59:55.661 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2020-04-10T06:59:55.663 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:55.702 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:55 +0000 (0:00:00.067) 0:01:06.155 ********** 2020-04-10T06:59:55.719 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* 2020-04-10T06:59:55.719 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "size": 4096, "state": "directory", "uid": 0} 2020-04-10T06:59:55.772 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:55 +0000 (0:00:00.069) 0:01:06.225 ********** 2020-04-10T06:59:56.092 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-04-10T06:59:56.093 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "9a447e184389aadb9aaf1e77f30f0a7f8b716c4e", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "size": 12725, "state": "file", "uid": 0} 2020-04-10T06:59:56.381 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "size": 8602, "state": "file", "uid": 0} 2020-04-10T06:59:56.568 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu/Fedora)] ********** 2020-04-10T06:59:56.570 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => {"cache_update_time": 1586501716, "cache_updated": false, "changed": false} 2020-04-10T06:59:56.608 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:56 +0000 (0:00:00.836) 0:01:07.061 ********** 2020-04-10T06:59:56.677 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-04-10T06:59:56.678 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "size": 3605, "state": "file", "uid": 0} 2020-04-10T06:59:56.739 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:56 +0000 (0:00:00.131) 0:01:07.192 ********** 2020-04-10T06:59:56.776 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2020-04-10T06:59:56.778 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib/nagios/plugins", "nrpe_group": "nagios", "nrpe_packages": ["nagios-nrpe-server", "nagios-plugins-basic"], "nrpe_service_name": "nagios-nrpe-server", "nrpe_user": "nagios"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/apt_systems.yml"], "changed": false} 2020-04-10T06:59:56.816 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:56 +0000 (0:00:00.077) 0:01:07.269 ********** 2020-04-10T06:59:56.844 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2020-04-10T06:59:56.846 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:56.885 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:56 +0000 (0:00:00.068) 0:01:07.338 ********** 2020-04-10T06:59:56.913 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* 2020-04-10T06:59:56.914 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-04-10T06:59:56.955 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:56 +0000 (0:00:00.070) 0:01:07.408 ********** 2020-04-10T06:59:56.961 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** 2020-04-10T06:59:56.962 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => {"changed": false, "checksum": "4880e187ff9f437cc33858eb1d0fa49a06d67be4", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "size": 180, "state": "file", "uid": 0} 2020-04-10T06:59:57.013 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:57 +0000 (0:00:00.057) 0:01:07.466 ********** 2020-04-10T06:59:57.020 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2020-04-10T06:59:57.025 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} skipping: [smithi204.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} skipping: [smithi204.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} skipping: [smithi204.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-04-10T06:59:57.067 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:57 +0000 (0:00:00.053) 0:01:07.520 ********** 2020-04-10T06:59:57.119 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:57.155 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:57 +0000 (0:00:00.088) 0:01:07.608 ********** 2020-04-10T06:59:57.329 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2020-04-10T06:59:57.332 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/nagios/plugins/check_mem.sh", "size": 1964, "state": "file", "uid": 0} 2020-04-10T06:59:57.371 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:57 +0000 (0:00:00.215) 0:01:07.824 ********** 2020-04-10T06:59:57.506 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ ok: [smithi101.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-04-10T06:59:57.542 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:57 +0000 (0:00:00.170) 0:01:07.995 ********** 2020-04-10T06:59:57.568 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2020-04-10T06:59:57.570 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:57.608 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:57 +0000 (0:00:00.065) 0:01:08.061 ********** 2020-04-10T06:59:57.634 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** 2020-04-10T06:59:57.636 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-04-10T06:59:57.675 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:57 +0000 (0:00:00.066) 0:01:08.127 ********** 2020-04-10T06:59:57.712 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2020-04-10T06:59:57.714 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:57.764 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:57 +0000 (0:00:00.090) 0:01:08.218 ********** 2020-04-10T06:59:58.097 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* ok: [smithi101.front.sepia.ceph.com] => {"changed": false, "checksum": "9c3d1d79583cf76a6d81bb2ec150bc4af5ca3c5c", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "size": 1166, "state": "file", "uid": 0} TASK [common : Install nrpe package and dependencies (Ubuntu/Fedora)] ********** 2020-04-10T06:59:58.098 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => {"cache_update_time": 1586501758, "cache_updated": false, "changed": false} 2020-04-10T06:59:58.140 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:58 +0000 (0:00:00.375) 0:01:08.593 ********** 2020-04-10T06:59:58.186 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:58 +0000 (0:00:00.045) 0:01:08.639 ********** 2020-04-10T06:59:58.530 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** 2020-04-10T06:59:58.531 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => {"changed": false, "checksum": "4880e187ff9f437cc33858eb1d0fa49a06d67be4", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "size": 180, "state": "file", "uid": 0} 2020-04-10T06:59:58.586 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:58 +0000 (0:00:00.400) 0:01:09.039 ********** 2020-04-10T06:59:58.781 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2020-04-10T06:59:58.784 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nagios-nrpe-server", "state": "started", "status": {"ActiveEnterTimestamp": "Fri 2020-04-10 06:58:38 UTC", "ActiveEnterTimestampMonotonic": "17618065", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "local-fs.target time-sync.target systemd-tmpfiles-setup.service systemd-journald.socket basic.target system.slice remote-fs.target network.target sysinit.target nss-lookup.target -.mount var-run.mount", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Fri 2020-04-10 06:58:38 UTC", "AssertTimestampMonotonic": "17617319", "Before": "xdm.service shutdown.target multi-user.target getty@tty1.service plymouth-quit.service", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Fri 2020-04-10 06:58:38 UTC", "ConditionTimestampMonotonic": "17617318", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target nrpe.socket", "ControlGroup": "/system.slice/nagios-nrpe-ser 2020-04-10T06:59:58.785 INFO:teuthology.task.ansible.out:ver.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "EnvironmentFile": "/etc/default/nagios-nrpe-server (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1283", "ExecMainStartTimestamp": "Fri 2020-04-10 06:58:38 UTC", "ExecMainStartTimestampMonotonic": "17618042", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_OPTS ; ignore_errors=no ; start_time=[Fri 2020-04-10 06:58:38 UTC] ; stop_time=[n/a] ; pid=1283 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nagios/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/nagios-nrpe-server.service", "GID": "118", "Group": "nagios", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "nagios-nrpe-server.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Fri 2020-04-10 06:58:38 UTC", "InactiveExitTimestampMonotonic": "17618065", "InvocationID": "3326b930d7b840e9951cbf50c99545b7", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "i 2020-04-10T06:59:58.786 INFO:teuthology.task.ansible.out:nfinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128031", "LimitNPROCSoft": "128031", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128031", "LimitSIGPENDINGSoft": "128031", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1283", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "nagios-nrpe-server.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/var/run/nagios/nrpe.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice sysinit.target -.mount", "RequiresMountsFor": "/var/tmp /tmp", "Restart": "on-abort", "RestartUSec": "100ms", 2020-04-10T06:59:58.788 INFO:teuthology.task.ansible.out: "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Fri 2020-04-10 06:58:38 UTC", "StateChangeTimestampMonotonic": "17618065", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "109", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "User": "nagios", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Fri 2020-04-10 06:58:38 UTC", "WatchdogTimestampMonotonic": "17618064", "WatchdogUSec": "0"}} 2020-04-10T06:59:58.829 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:58 +0000 (0:00:00.242) 0:01:09.282 ********** 2020-04-10T06:59:58.857 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* 2020-04-10T06:59:58.857 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:58.910 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:58 +0000 (0:00:00.081) 0:01:09.363 ********** 2020-04-10T06:59:58.931 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2020-04-10T06:59:58.932 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/nagios/plugins/check_mem.sh", "size": 1964, "state": "file", "uid": 0} 2020-04-10T06:59:58.987 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:58 +0000 (0:00:00.076) 0:01:09.440 ********** 2020-04-10T06:59:58.992 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** 2020-04-10T06:59:58.994 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:59.037 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:59 +0000 (0:00:00.050) 0:01:09.490 ********** 2020-04-10T06:59:59.065 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* 2020-04-10T06:59:59.066 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:59.106 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:59 +0000 (0:00:00.068) 0:01:09.559 ********** 2020-04-10T06:59:59.132 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2020-04-10T06:59:59.133 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-04-10T06:59:59.173 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:59 +0000 (0:00:00.067) 0:01:09.626 ********** 2020-04-10T06:59:59.179 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** 2020-04-10T06:59:59.180 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:59.223 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:59 +0000 (0:00:00.049) 0:01:09.676 ********** 2020-04-10T06:59:59.228 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2020-04-10T06:59:59.230 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:59.271 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:59 +0000 (0:00:00.047) 0:01:09.724 ********** 2020-04-10T06:59:59.277 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** 2020-04-10T06:59:59.278 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:59.323 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:59 +0000 (0:00:00.051) 0:01:09.776 ********** 2020-04-10T06:59:59.329 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** skipping: [smithi204.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-04-10T06:59:59.377 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:59 +0000 (0:00:00.054) 0:01:09.830 ********** 2020-04-10T06:59:59.384 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** 2020-04-10T06:59:59.386 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:59.433 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:59 +0000 (0:00:00.056) 0:01:09.886 ********** 2020-04-10T06:59:59.439 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2020-04-10T06:59:59.443 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:59.491 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:59 +0000 (0:00:00.057) 0:01:09.944 ********** 2020-04-10T06:59:59.497 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** 2020-04-10T06:59:59.498 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:59.539 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:59 +0000 (0:00:00.047) 0:01:09.992 ********** 2020-04-10T06:59:59.575 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** 2020-04-10T06:59:59.577 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:59.615 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:59 +0000 (0:00:00.076) 0:01:10.068 ********** 2020-04-10T06:59:59.644 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:59.696 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:59 +0000 (0:00:00.081) 0:01:10.149 ********** 2020-04-10T06:59:59.729 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T06:59:59.770 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:59 +0000 (0:00:00.073) 0:01:10.223 ********** 2020-04-10T06:59:59.807 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2020-04-10T06:59:59.807 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => {"ansible_facts": {"apt_repos": [], "ceph_packages_to_remove": ["ceph", "ceph-common", "libcephfs1", "radosgw", "python-ceph", "python-rados", "python-cephfs", "python-rbd", "librbd1", "librados2", "ceph-fs-common-dbg", "ceph-fs-common"], "common_apt_repos": [], "common_packages": [], "nfs_service": "nfs-kernel-server", "ntp_service_name": "ntp", "packages": [], "packages_to_remove": ["multipath-tools"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "ssh"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/apt_systems.yml"], "changed": false} 2020-04-10T06:59:59.873 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:59 +0000 (0:00:00.102) 0:01:10.326 ********** 2020-04-10T06:59:59.881 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* 2020-04-10T06:59:59.883 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => {"changed": false, "checksum": "9c3d1d79583cf76a6d81bb2ec150bc4af5ca3c5c", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "size": 1166, "state": "file", "uid": 0} 2020-04-10T06:59:59.927 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:59 +0000 (0:00:00.054) 0:01:10.380 ********** 2020-04-10T06:59:59.933 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2020-04-10T06:59:59.934 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml) => {"ansible_facts": {"common_packages": ["libfcgi0ldbl", "python-virtualenv", "python-dev", "libev-dev", "perl", "libwww-perl", "lsb-release", "build-essential", "sysstat", "gdb", "python-configobj", "python-gevent", "libedit2", "cryptsetup-bin", "xfsprogs", "gdisk", "parted", "libuuid1", "btrfs-tools", "libatomic-ops-dev", "git-core", "attr", "dbench", "bonnie++", "valgrind", "python-nose", "ant", "libtool", "automake", "gettext", "uuid-dev", "libacl1-dev", "bc", "xfsdump", "xfslibs-dev", "libattr1-dev", "quota", "libcap2-bin", "libncurses5-dev", "lvm2", "vim", "pdsh", "collectl", "blktrace", "python-numpy", "python-matplotlib", "genisoimage", "libjson-xs-perl", "xml-twig-tools", "default-jdk", "junit4", "tgt", "open-iscsi", "cifs-utils", "ipcalc", "nfs-common", "nfs-kernel-server", "software-properties-common"], "non_aarch64_common_packages": ["smbios-utils", "libfcgi", "sysprof"], "non_aarch64_packages_to_upgrade": ["libapache2-mod-fastcgi"], "packages_to_remove": ["openmpi-common"], "packages_to_upgrade": ["apt", "apache2"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"} 2020-04-10T06:59:59.978 INFO:teuthology.task.ansible.out:Friday 10 April 2020 06:59:59 +0000 (0:00:00.050) 0:01:10.431 ********** 2020-04-10T07:00:00.015 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2020-04-10T07:00:00.017 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml) => {"ansible_facts": {"non_aarch64_packages": ["libgoogle-perftools4", "iozone3", "libssl1.0.0"], "non_aarch64_packages_to_upgrade": [], "packages": ["mpich", "qemu-system-x86", "lttng-tools", "libtool-bin", "docker.io", "python3-nose"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"} 2020-04-10T07:00:00.058 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:00 +0000 (0:00:00.080) 0:01:10.511 ********** 2020-04-10T07:00:00.095 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2020-04-10T07:00:00.097 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2020-04-10T07:00:00.136 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:00 +0000 (0:00:00.077) 0:01:10.589 ********** 2020-04-10T07:00:00.166 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2020-04-10T07:00:00.167 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nagios-nrpe-server", "state": "started", "status": {"ActiveEnterTimestamp": "Fri 2020-04-10 06:58:38 UTC", "ActiveEnterTimestampMonotonic": "18179783", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "systemd-journald.socket time-sync.target nss-lookup.target sysinit.target system.slice -.mount network.target var-run.mount local-fs.target remote-fs.target systemd-tmpfiles-setup.service basic.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Fri 2020-04-10 06:58:38 UTC", "AssertTimestampMonotonic": "18179101", "Before": "xdm.service shutdown.target multi-user.target getty@tty1.service plymouth-quit.service", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Fri 2020-04-10 06:58:38 UTC", "ConditionTimestampMonotonic": "18179100", "ConfigurationDirectoryMode": "0755", "Conflicts": "nrpe.socket shutdown.target", "ControlGroup": "/system.slice/nagios-nrpe-ser 2020-04-10T07:00:00.169 INFO:teuthology.task.ansible.out:ver.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "EnvironmentFile": "/etc/default/nagios-nrpe-server (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1135", "ExecMainStartTimestamp": "Fri 2020-04-10 06:58:38 UTC", "ExecMainStartTimestampMonotonic": "18179751", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_OPTS ; ignore_errors=no ; start_time=[Fri 2020-04-10 06:58:38 UTC] ; stop_time=[n/a] ; pid=1135 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nagios/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/nagios-nrpe-server.service", "GID": "118", "Group": "nagios", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "nagios-nrpe-server.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Fri 2020-04-10 06:58:38 UTC", "InactiveExitTimestampMonotonic": "18179783", "InvocationID": "5a83a7ca0f054fcabba35bbc825447e8", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "i 2020-04-10T07:00:00.170 INFO:teuthology.task.ansible.out:nfinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128031", "LimitNPROCSoft": "128031", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128031", "LimitSIGPENDINGSoft": "128031", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1135", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "nagios-nrpe-server.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/var/run/nagios/nrpe.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target system.slice -.mount", "RequiresMountsFor": "/var/tmp /tmp", "Restart": "on-abort", "RestartUSec": "100ms", 2020-04-10T07:00:00.172 INFO:teuthology.task.ansible.out: "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Fri 2020-04-10 06:58:38 UTC", "StateChangeTimestampMonotonic": "18179783", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "109", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "User": "nagios", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Fri 2020-04-10 06:58:38 UTC", "WatchdogTimestampMonotonic": "18179782", "WatchdogUSec": "0"}} 2020-04-10T07:00:00.209 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:00 +0000 (0:00:00.072) 0:01:10.662 ********** 2020-04-10T07:00:00.235 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* 2020-04-10T07:00:00.236 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:00.276 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:00 +0000 (0:00:00.067) 0:01:10.729 ********** 2020-04-10T07:00:00.281 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** 2020-04-10T07:00:00.282 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2020-04-10T07:00:00.324 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:00 +0000 (0:00:00.048) 0:01:10.777 ********** 2020-04-10T07:00:00.331 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:00.381 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:00 +0000 (0:00:00.048) 0:01:10.826 ********** 2020-04-10T07:00:00.410 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:00.452 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:00 +0000 (0:00:00.078) 0:01:10.905 ********** 2020-04-10T07:00:00.458 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2020-04-10T07:00:00.459 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => {"changed": false, "gid": 1211, "name": "ubuntu", "state": "present", "system": false} 2020-04-10T07:00:00.499 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:00 +0000 (0:00:00.047) 0:01:10.952 ********** 2020-04-10T07:00:00.505 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** 2020-04-10T07:00:00.506 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:00.551 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:00 +0000 (0:00:00.050) 0:01:11.003 ********** 2020-04-10T07:00:00.555 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** 2020-04-10T07:00:00.555 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:00.595 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:00 +0000 (0:00:00.045) 0:01:11.049 ********** 2020-04-10T07:00:00.601 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** 2020-04-10T07:00:00.603 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:00.641 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:00 +0000 (0:00:00.045) 0:01:11.094 ********** 2020-04-10T07:00:00.645 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2020-04-10T07:00:00.646 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:00.689 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:00 +0000 (0:00:00.048) 0:01:11.142 ********** 2020-04-10T07:00:00.694 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** 2020-04-10T07:00:00.696 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:00.738 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:00 +0000 (0:00:00.048) 0:01:11.191 ********** 2020-04-10T07:00:00.763 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** 2020-04-10T07:00:00.765 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:00.806 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:00 +0000 (0:00:00.068) 0:01:11.259 ********** 2020-04-10T07:00:00.843 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:00.881 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:00 +0000 (0:00:00.074) 0:01:11.334 ********** 2020-04-10T07:00:00.886 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2020-04-10T07:00:00.887 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/sh", "state": "present", "uid": 10101} 2020-04-10T07:00:00.928 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:00 +0000 (0:00:00.046) 0:01:11.381 ********** 2020-04-10T07:00:00.935 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** 2020-04-10T07:00:00.937 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:00.980 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:00 +0000 (0:00:00.051) 0:01:11.433 ********** 2020-04-10T07:00:01.005 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** 2020-04-10T07:00:01.006 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:01.045 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:01 +0000 (0:00:00.065) 0:01:11.498 ********** 2020-04-10T07:00:01.071 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2020-04-10T07:00:01.072 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => {"ansible_facts": {"apt_repos": [], "ceph_packages_to_remove": ["ceph", "ceph-common", "libcephfs1", "radosgw", "python-ceph", "python-rados", "python-cephfs", "python-rbd", "librbd1", "librados2", "ceph-fs-common-dbg", "ceph-fs-common"], "common_apt_repos": [], "common_packages": [], "nfs_service": "nfs-kernel-server", "ntp_service_name": "ntp", "packages": [], "packages_to_remove": ["multipath-tools"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "ssh"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/apt_systems.yml"], "changed": false} 2020-04-10T07:00:01.114 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:01 +0000 (0:00:00.068) 0:01:11.567 ********** 2020-04-10T07:00:01.150 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2020-04-10T07:00:01.151 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml) => {"ansible_facts": {"common_packages": ["libfcgi0ldbl", "python-virtualenv", "python-dev", "libev-dev", "perl", "libwww-perl", "lsb-release", "build-essential", "sysstat", "gdb", "python-configobj", "python-gevent", "libedit2", "cryptsetup-bin", "xfsprogs", "gdisk", "parted", "libuuid1", "btrfs-tools", "libatomic-ops-dev", "git-core", "attr", "dbench", "bonnie++", "valgrind", "python-nose", "ant", "libtool", "automake", "gettext", "uuid-dev", "libacl1-dev", "bc", "xfsdump", "xfslibs-dev", "libattr1-dev", "quota", "libcap2-bin", "libncurses5-dev", "lvm2", "vim", "pdsh", "collectl", "blktrace", "python-numpy", "python-matplotlib", "genisoimage", "libjson-xs-perl", "xml-twig-tools", "default-jdk", "junit4", "tgt", "open-iscsi", "cifs-utils", "ipcalc", "nfs-common", "nfs-kernel-server", "software-properties-common"], "non_aarch64_common_packages": ["smbios-utils", "libfcgi", "sysprof"], "non_aarch64_packages_to_upgrade": ["libapache2-mod-fastcgi"], "packages_to_remove": ["openmpi-common"], "packages_to_upgrade": ["apt", "apache2"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"} 2020-04-10T07:00:01.195 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:01 +0000 (0:00:00.079) 0:01:11.647 ********** 2020-04-10T07:00:01.240 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2020-04-10T07:00:01.241 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml) => {"ansible_facts": {"non_aarch64_packages": ["libgoogle-perftools4", "iozone3", "libssl1.0.0"], "non_aarch64_packages_to_upgrade": [], "packages": ["mpich", "qemu-system-x86", "lttng-tools", "libtool-bin", "docker.io", "python3-nose"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"} 2020-04-10T07:00:01.284 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:01 +0000 (0:00:00.087) 0:01:11.735 ********** 2020-04-10T07:00:01.330 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2020-04-10T07:00:01.331 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2020-04-10T07:00:01.375 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:01 +0000 (0:00:00.093) 0:01:11.828 ********** 2020-04-10T07:00:01.385 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** ok: [smithi101.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "size": 112, "state": "file", "uid": 0} 2020-04-10T07:00:01.427 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:01 +0000 (0:00:00.051) 0:01:11.880 ********** 2020-04-10T07:00:01.466 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2020-04-10T07:00:01.467 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi101"}, "changed": false} 2020-04-10T07:00:01.506 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:01 +0000 (0:00:00.079) 0:01:11.959 ********** 2020-04-10T07:00:01.523 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** 2020-04-10T07:00:01.524 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2020-04-10T07:00:01.572 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:01 +0000 (0:00:00.065) 0:01:12.025 ********** 2020-04-10T07:00:01.709 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2020-04-10T07:00:01.710 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => {"changed": false, "gid": 1211, "name": "ubuntu", "state": "present", "system": false} 2020-04-10T07:00:01.771 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:01 +0000 (0:00:00.198) 0:01:12.224 ********** 2020-04-10T07:00:01.797 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** 2020-04-10T07:00:01.799 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:01.856 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:01 +0000 (0:00:00.083) 0:01:12.307 ********** 2020-04-10T07:00:01.881 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2020-04-10T07:00:01.882 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:01.932 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:01 +0000 (0:00:00.077) 0:01:12.385 ********** 2020-04-10T07:00:02.097 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2020-04-10T07:00:02.097 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/sh", "state": "present", "uid": 10101} 2020-04-10T07:00:02.138 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:02 +0000 (0:00:00.205) 0:01:12.590 ********** 2020-04-10T07:00:02.531 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ changed: [smithi101.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "front.sepia.ceph.com", "ansible_fqdn": "smithi101.front.sepia.ceph.com", "ansible_hostname": "smithi101", "ansible_nodename": "smithi101"}, "changed": true, "name": "smithi101"} 2020-04-10T07:00:02.586 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:02 +0000 (0:00:00.448) 0:01:13.039 ********** 2020-04-10T07:00:02.594 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2020-04-10T07:00:02.597 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "size": 112, "state": "file", "uid": 0} 2020-04-10T07:00:02.652 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:02 +0000 (0:00:00.065) 0:01:13.105 ********** 2020-04-10T07:00:02.690 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2020-04-10T07:00:02.692 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi204"}, "changed": false} 2020-04-10T07:00:02.739 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:02 +0000 (0:00:00.086) 0:01:13.192 ********** 2020-04-10T07:00:02.929 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* 2020-04-10T07:00:02.930 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => {"changed": false, "checksum": "2f7bfea2455a0d9220592aa0c21ac69c312e7471", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "size": 2646, "state": "file", "uid": 0} 2020-04-10T07:00:02.980 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:02 +0000 (0:00:00.240) 0:01:13.433 ********** 2020-04-10T07:00:03.193 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ 2020-04-10T07:00:03.195 INFO:teuthology.task.ansible.out:changed: [smithi204.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "front.sepia.ceph.com", "ansible_fqdn": "smithi204.front.sepia.ceph.com", "ansible_hostname": "smithi204", "ansible_nodename": "smithi204"}, "changed": true, "name": "smithi204"} 2020-04-10T07:00:03.236 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:03 +0000 (0:00:00.255) 0:01:13.689 ********** 2020-04-10T07:00:03.340 INFO:teuthology.task.ansible.out: TASK [testnode : Upload ssh_config] ******************************************** 2020-04-10T07:00:03.341 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "size": 570, "state": "file", "uid": 0} 2020-04-10T07:00:03.382 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:03 +0000 (0:00:00.146) 0:01:13.835 ********** 2020-04-10T07:00:03.572 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* ok: [smithi204.front.sepia.ceph.com] => {"changed": false, "checksum": "2f7bfea2455a0d9220592aa0c21ac69c312e7471", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "size": 2646, "state": "file", "uid": 0} 2020-04-10T07:00:03.628 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:03 +0000 (0:00:00.245) 0:01:14.081 ********** 2020-04-10T07:00:03.907 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** 2020-04-10T07:00:03.909 INFO:teuthology.task.ansible.out:changed: [smithi101.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2020-04-10T07:00:03.954 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:03 +0000 (0:00:00.326) 0:01:14.407 ********** 2020-04-10T07:00:03.981 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* 2020-04-10T07:00:03.983 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:04.029 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:04 +0000 (0:00:00.075) 0:01:14.482 ********** 2020-04-10T07:00:04.041 INFO:teuthology.task.ansible.out: TASK [testnode : Upload ssh_config] ******************************************** ok: [smithi204.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "size": 570, "state": "file", "uid": 0} 2020-04-10T07:00:04.080 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:04 +0000 (0:00:00.050) 0:01:14.533 ********** 2020-04-10T07:00:04.085 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2020-04-10T07:00:04.087 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:04.126 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:04 +0000 (0:00:00.045) 0:01:14.579 ********** 2020-04-10T07:00:04.162 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2020-04-10T07:00:04.164 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:04.204 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:04 +0000 (0:00:00.077) 0:01:14.657 ********** 2020-04-10T07:00:04.235 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:04.281 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:04 +0000 (0:00:00.077) 0:01:14.734 ********** 2020-04-10T07:00:04.324 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2020-04-10T07:00:04.325 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:04.375 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:04 +0000 (0:00:00.093) 0:01:14.828 ********** 2020-04-10T07:00:04.415 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:04.465 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:04 +0000 (0:00:00.090) 0:01:14.918 ********** 2020-04-10T07:00:04.492 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2020-04-10T07:00:04.493 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:04.533 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:04 +0000 (0:00:00.068) 0:01:14.987 ********** 2020-04-10T07:00:04.573 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** 2020-04-10T07:00:04.574 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:04.636 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:04 +0000 (0:00:00.102) 0:01:15.089 ********** 2020-04-10T07:00:04.643 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** 2020-04-10T07:00:04.644 INFO:teuthology.task.ansible.out:changed: [smithi204.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2020-04-10T07:00:04.688 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:04 +0000 (0:00:00.052) 0:01:15.141 ********** 2020-04-10T07:00:04.695 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2020-04-10T07:00:04.740 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:04 +0000 (0:00:00.051) 0:01:15.193 ********** 2020-04-10T07:00:04.747 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* 2020-04-10T07:00:04.748 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:04.793 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:04 +0000 (0:00:00.052) 0:01:15.246 ********** 2020-04-10T07:00:04.798 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2020-04-10T07:00:04.842 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:04 +0000 (0:00:00.049) 0:01:15.295 ********** 2020-04-10T07:00:04.848 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2020-04-10T07:00:04.849 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:04.893 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:04 +0000 (0:00:00.050) 0:01:15.346 ********** 2020-04-10T07:00:04.898 INFO:teuthology.task.ansible.out: TASK [testnode : Enable copr repos] ******************************************** 2020-04-10T07:00:04.940 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:04 +0000 (0:00:00.047) 0:01:15.393 ********** 2020-04-10T07:00:04.946 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2020-04-10T07:00:04.947 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:04.988 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:04 +0000 (0:00:00.047) 0:01:15.441 ********** 2020-04-10T07:00:04.993 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** 2020-04-10T07:00:04.995 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:05.036 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:05 +0000 (0:00:00.048) 0:01:15.489 ********** 2020-04-10T07:00:05.043 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2020-04-10T07:00:05.044 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:05.089 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:05 +0000 (0:00:00.052) 0:01:15.542 ********** 2020-04-10T07:00:05.096 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* skipping: [smithi101.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} skipping: [smithi101.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2020-04-10T07:00:05.137 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:05 +0000 (0:00:00.048) 0:01:15.591 ********** 2020-04-10T07:00:05.144 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2020-04-10T07:00:05.145 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:05.193 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:05 +0000 (0:00:00.055) 0:01:15.646 ********** 2020-04-10T07:00:05.198 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2020-04-10T07:00:05.200 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:05.242 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:05 +0000 (0:00:00.049) 0:01:15.695 ********** 2020-04-10T07:00:05.249 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:05.288 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:05 +0000 (0:00:00.046) 0:01:15.741 ********** 2020-04-10T07:00:05.293 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-04-10T07:00:05.296 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:05.335 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:05 +0000 (0:00:00.046) 0:01:15.788 ********** 2020-04-10T07:00:05.342 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:05.387 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:05 +0000 (0:00:00.051) 0:01:15.840 ********** 2020-04-10T07:00:05.392 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* 2020-04-10T07:00:05.393 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:05.448 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:05 +0000 (0:00:00.061) 0:01:15.901 ********** 2020-04-10T07:00:05.459 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:05.495 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:05 +0000 (0:00:00.046) 0:01:15.948 ********** 2020-04-10T07:00:05.500 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** 2020-04-10T07:00:05.501 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:05.543 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:05 +0000 (0:00:00.048) 0:01:15.996 ********** 2020-04-10T07:00:05.549 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2020-04-10T07:00:05.591 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:05 +0000 (0:00:00.047) 0:01:16.044 ********** 2020-04-10T07:00:05.596 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2020-04-10T07:00:05.598 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:05.641 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:05 +0000 (0:00:00.050) 0:01:16.094 ********** 2020-04-10T07:00:05.649 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2020-04-10T07:00:05.687 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:05 +0000 (0:00:00.046) 0:01:16.141 ********** 2020-04-10T07:00:05.694 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:05.742 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:05 +0000 (0:00:00.054) 0:01:16.195 ********** 2020-04-10T07:00:05.744 INFO:teuthology.task.ansible.out: 2020-04-10T07:00:05.749 INFO:teuthology.task.ansible.out: 2020-04-10T07:00:05.751 INFO:teuthology.task.ansible.out:TASK [testnode : Enable copr repos] ******************************************** 2020-04-10T07:00:05.795 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:05 +0000 (0:00:00.052) 0:01:16.248 ********** 2020-04-10T07:00:05.801 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2020-04-10T07:00:05.802 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:05.855 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:05 +0000 (0:00:00.060) 0:01:16.309 ********** 2020-04-10T07:00:05.862 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** 2020-04-10T07:00:05.863 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:05.909 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:05 +0000 (0:00:00.053) 0:01:16.362 ********** 2020-04-10T07:00:05.915 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2020-04-10T07:00:05.916 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:05.964 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:05 +0000 (0:00:00.054) 0:01:16.416 ********** 2020-04-10T07:00:05.969 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2020-04-10T07:00:05.971 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} skipping: [smithi204.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2020-04-10T07:00:06.028 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:06 +0000 (0:00:00.060) 0:01:16.477 ********** 2020-04-10T07:00:06.029 INFO:teuthology.task.ansible.out: TASK [testnode : Base group workaround for BZ1782899] ************************** 2020-04-10T07:00:06.029 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:06.073 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:06 +0000 (0:00:00.048) 0:01:16.525 ********** 2020-04-10T07:00:06.078 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2020-04-10T07:00:06.080 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:06.120 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:06 +0000 (0:00:00.047) 0:01:16.573 ********** 2020-04-10T07:00:06.125 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-04-10T07:00:06.126 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:06.169 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:06 +0000 (0:00:00.049) 0:01:16.622 ********** 2020-04-10T07:00:06.175 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-04-10T07:00:06.177 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:06.222 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:06 +0000 (0:00:00.053) 0:01:16.675 ********** 2020-04-10T07:00:06.228 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2020-04-10T07:00:06.229 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:06.296 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:06 +0000 (0:00:00.074) 0:01:16.749 ********** 2020-04-10T07:00:06.306 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* 2020-04-10T07:00:06.307 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:06.352 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:06 +0000 (0:00:00.055) 0:01:16.805 ********** 2020-04-10T07:00:06.357 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2020-04-10T07:00:06.359 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:06.400 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:06 +0000 (0:00:00.048) 0:01:16.854 ********** 2020-04-10T07:00:06.407 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** 2020-04-10T07:00:06.409 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:06.452 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:06 +0000 (0:00:00.048) 0:01:16.902 ********** 2020-04-10T07:00:06.454 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-04-10T07:00:06.455 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:06.504 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:06 +0000 (0:00:00.054) 0:01:16.957 ********** 2020-04-10T07:00:06.510 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2020-04-10T07:00:06.517 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:06.559 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:06 +0000 (0:00:00.055) 0:01:17.012 ********** 2020-04-10T07:00:06.565 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:06.606 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:06 +0000 (0:00:00.046) 0:01:17.059 ********** 2020-04-10T07:00:06.612 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2020-04-10T07:00:06.613 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:06.659 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:06 +0000 (0:00:00.051) 0:01:17.111 ********** 2020-04-10T07:00:06.664 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2020-04-10T07:00:06.665 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:06.705 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:06 +0000 (0:00:00.047) 0:01:17.158 ********** 2020-04-10T07:00:06.711 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2020-04-10T07:00:06.713 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:06.754 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:06 +0000 (0:00:00.049) 0:01:17.207 ********** 2020-04-10T07:00:06.769 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:06.804 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:06 +0000 (0:00:00.049) 0:01:17.257 ********** 2020-04-10T07:00:06.810 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2020-04-10T07:00:06.811 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:06.850 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:06 +0000 (0:00:00.046) 0:01:17.303 ********** 2020-04-10T07:00:06.877 INFO:teuthology.task.ansible.out: TASK [testnode : Base group workaround for BZ1782899] ************************** 2020-04-10T07:00:06.878 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:06.926 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:06 +0000 (0:00:00.075) 0:01:17.379 ********** 2020-04-10T07:00:06.941 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** 2020-04-10T07:00:06.942 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => {"changed": false, "cmd": "ls -1 /etc/apt/sources.list.d/", "delta": "0:00:00.005177", "end": "2020-04-10 07:00:06.925855", "rc": 0, "start": "2020-04-10 07:00:06.920678", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-04-10T07:00:06.986 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:06 +0000 (0:00:00.060) 0:01:17.439 ********** 2020-04-10T07:00:06.992 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-04-10T07:00:06.993 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:07.041 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:07 +0000 (0:00:00.054) 0:01:17.494 ********** 2020-04-10T07:00:07.047 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2020-04-10T07:00:07.090 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:07 +0000 (0:00:00.048) 0:01:17.543 ********** 2020-04-10T07:00:07.096 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2020-04-10T07:00:07.097 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:07.137 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:07 +0000 (0:00:00.046) 0:01:17.590 ********** 2020-04-10T07:00:07.163 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2020-04-10T07:00:07.165 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:07.205 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:07 +0000 (0:00:00.068) 0:01:17.658 ********** 2020-04-10T07:00:07.232 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-04-10T07:00:07.234 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:07.279 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:07 +0000 (0:00:00.066) 0:01:17.725 ********** 2020-04-10T07:00:07.301 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:07.343 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:07 +0000 (0:00:00.071) 0:01:17.796 ********** 2020-04-10T07:00:07.369 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2020-04-10T07:00:07.371 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:07.410 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:07 +0000 (0:00:00.067) 0:01:17.864 ********** 2020-04-10T07:00:07.437 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** 2020-04-10T07:00:07.439 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:07.501 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:07 +0000 (0:00:00.090) 0:01:17.954 ********** 2020-04-10T07:00:07.510 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2020-04-10T07:00:07.512 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => {"changed": false, "checksum": "ba101482124806c7f6ea9fc7b4acfda303391ad6", "dest": "/etc/apt/preferences.d/ceph.pref", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/apt/preferences.d/ceph.pref", "size": 52, "state": "file", "uid": 0} 2020-04-10T07:00:07.563 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:07 +0000 (0:00:00.060) 0:01:18.016 ********** 2020-04-10T07:00:07.591 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2020-04-10T07:00:07.592 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:07.637 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:07 +0000 (0:00:00.074) 0:01:18.090 ********** 2020-04-10T07:00:07.645 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** 2020-04-10T07:00:07.646 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => {"changed": false, "cmd": "ls -1 /etc/apt/sources.list.d/", "delta": "0:00:00.005149", "end": "2020-04-10 07:00:07.845268", "rc": 0, "start": "2020-04-10 07:00:07.840119", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-04-10T07:00:07.689 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:07 +0000 (0:00:00.051) 0:01:18.142 ********** 2020-04-10T07:00:07.717 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2020-04-10T07:00:07.761 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:07 +0000 (0:00:00.072) 0:01:18.214 ********** 2020-04-10T07:00:08.114 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2020-04-10T07:00:08.115 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => {"changed": false, "checksum": "ba101482124806c7f6ea9fc7b4acfda303391ad6", "dest": "/etc/apt/preferences.d/ceph.pref", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/apt/preferences.d/ceph.pref", "size": 52, "state": "file", "uid": 0} 2020-04-10T07:00:08.162 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:08 +0000 (0:00:00.400) 0:01:18.615 ********** 2020-04-10T07:00:08.189 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2020-04-10T07:00:08.191 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:08.246 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:08 +0000 (0:00:00.084) 0:01:18.699 ********** 2020-04-10T07:00:08.978 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* ok: [smithi101.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc"} 2020-04-10T07:00:09.267 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc"} 2020-04-10T07:00:09.935 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc"} 2020-04-10T07:00:09.984 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:09 +0000 (0:00:01.731) 0:01:20.431 ********** 2020-04-10T07:00:10.216 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc"} 2020-04-10T07:00:10.275 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:10 +0000 (0:00:00.296) 0:01:20.728 ********** 2020-04-10T07:00:10.895 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2020-04-10T07:00:10.896 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => {"cache_update_time": 1586501716, "cache_updated": false, "changed": false} 2020-04-10T07:00:10.941 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:10 +0000 (0:00:00.666) 0:01:21.394 ********** 2020-04-10T07:00:10.969 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2020-04-10T07:00:11.010 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:11 +0000 (0:00:00.069) 0:01:21.463 ********** 2020-04-10T07:00:11.181 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2020-04-10T07:00:11.182 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => {"cache_update_time": 1586501758, "cache_updated": false, "changed": false} 2020-04-10T07:00:11.183 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** [WARNING]: Consider using 'become', 'become_method', and 'become_user' rather than running sudo changed: [smithi101.front.sepia.ceph.com] => {"changed": true, "cmd": ["sudo", "apt-get", "clean"], "delta": "0:00:00.032381", "end": "2020-04-10 07:00:11.155547", "rc": 0, "start": "2020-04-10 07:00:11.123166", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-04-10T07:00:11.241 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:11 +0000 (0:00:00.230) 0:01:21.694 ********** 2020-04-10T07:00:11.281 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:11 +0000 (0:00:00.036) 0:01:21.731 ********** 2020-04-10T07:00:11.313 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2020-04-10T07:00:11.360 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:11 +0000 (0:00:00.082) 0:01:21.814 ********** 2020-04-10T07:00:11.514 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** 2020-04-10T07:00:11.515 INFO:teuthology.task.ansible.out:changed: [smithi204.front.sepia.ceph.com] => {"changed": true, "cmd": ["sudo", "apt-get", "clean"], "delta": "0:00:00.026206", "end": "2020-04-10 07:00:11.718189", "rc": 0, "start": "2020-04-10 07:00:11.691983", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-04-10T07:00:11.569 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:11 +0000 (0:00:00.208) 0:01:22.022 ********** 2020-04-10T07:00:14.227 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** 2020-04-10T07:00:14.228 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => {"cache_update_time": 1586501716, "cache_updated": false, "changed": false} 2020-04-10T07:00:14.273 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:14 +0000 (0:00:02.703) 0:01:24.726 ********** 2020-04-10T07:00:14.481 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => {"cache_update_time": 1586501758, "cache_updated": false, "changed": false} 2020-04-10T07:00:14.566 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:14 +0000 (0:00:00.292) 0:01:25.019 ********** 2020-04-10T07:00:24.277 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** changed: [smithi204.front.sepia.ceph.com] => {"changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good\n gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0\n libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1\n libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0\n libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0\n libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3\n libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0\n libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5\n libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following packages will be REMOVED:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 0 newly installed, 6 to remove and 48 not upgraded.\nAfter this operation, 47.5 MB disk space will be freed.\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database .. 2020-04-10T07:00:24.279 INFO:teuthology.task.ansible.out:. 95%\r(Reading database ... 100%\r(Reading database ... 137474 files and directories currently installed.)\r\nRemoving qemu-utils (1:2.11+dfsg-1ubuntu7.23) ...\r\nRemoving qemu-system-x86 (1:2.11+dfsg-1ubuntu7.23) ...\r\nRemoving qemu-system-common (1:2.11+dfsg-1ubuntu7.23) ...\r\nRemoving qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.23) ...\r\nRemoving librbd1 (12.2.12-0ubuntu0.18.04.4) ...\r\nRemoving librados2 (12.2.12-0ubuntu0.18.04.4) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu1) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer required:", " cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good", " gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0", " libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1", " libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0", " libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0", " libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3", " libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0", " libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5", " libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following packages will be REMOVED:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 0 newly installed, 6 to remove and 48 not upgraded.", "After this operation, 47.5 MB disk space will be freed.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading d 2020-04-10T07:00:24.281 INFO:teuthology.task.ansible.out:atabase ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137474 files and directories currently installed.)", "Removing qemu-utils (1:2.11+dfsg-1ubuntu7.23) ...", "Removing qemu-system-x86 (1:2.11+dfsg-1ubuntu7.23) ...", "Removing qemu-system-common (1:2.11+dfsg-1ubuntu7.23) ...", "Removing qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.23) ...", "Removing librbd1 (12.2.12-0ubuntu0.18.04.4) ...", "Removing librados2 (12.2.12-0ubuntu0.18.04.4) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1) ..."]} 2020-04-10T07:00:24.369 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:24 +0000 (0:00:09.795) 0:01:34.815 ********** 2020-04-10T07:00:24.539 INFO:teuthology.task.ansible.out:changed: [smithi101.front.sepia.ceph.com] => {"changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good\n gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0\n libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1\n libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0\n libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0\n libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3\n libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0\n libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5\n libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following packages will be REMOVED:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 0 newly installed, 6 to remove and 48 not upgraded.\nAfter this operation, 47.5 MB disk space will be freed.\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137474 files and directorie 2020-04-10T07:00:24.541 INFO:teuthology.task.ansible.out:s currently installed.)\r\nRemoving qemu-utils (1:2.11+dfsg-1ubuntu7.23) ...\r\nRemoving qemu-system-x86 (1:2.11+dfsg-1ubuntu7.23) ...\r\nRemoving qemu-system-common (1:2.11+dfsg-1ubuntu7.23) ...\r\nRemoving qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.23) ...\r\nRemoving librbd1 (12.2.12-0ubuntu0.18.04.4) ...\r\nRemoving librados2 (12.2.12-0ubuntu0.18.04.4) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu1) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer required:", " cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good", " gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0", " libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1", " libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0", " libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0", " libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3", " libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0", " libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5", " libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following packages will be REMOVED:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 0 newly installed, 6 to remove and 48 not upgraded.", "After this operation, 47.5 MB disk space will be freed.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Readin 2020-04-10T07:00:24.542 INFO:teuthology.task.ansible.out:g database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137474 files and directories currently installed.)", "Removing qemu-utils (1:2.11+dfsg-1ubuntu7.23) ...", "Removing qemu-system-x86 (1:2.11+dfsg-1ubuntu7.23) ...", "Removing qemu-system-common (1:2.11+dfsg-1ubuntu7.23) ...", "Removing qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.23) ...", "Removing librbd1 (12.2.12-0ubuntu0.18.04.4) ...", "Removing librados2 (12.2.12-0ubuntu0.18.04.4) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1) ..."]} 2020-04-10T07:00:24.597 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:24 +0000 (0:00:00.235) 0:01:35.050 ********** 2020-04-10T07:00:39.715 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-04-10T07:00:39.717 INFO:teuthology.task.ansible.out:changed: [smithi204.front.sepia.ceph.com] => {"cache_update_time": 1586501758, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good\n gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0\n libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1\n libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0\n libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0\n libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3\n libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0\n libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5\n libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following additional packages will be installed:\n apache2-bin apache2-data apache2-utils\nSuggested packages:\n www-browser apache2-doc apache2-suexec-pristine | apache2-suexec-custom\nThe following packages will be upgraded:\n apache2 apache2-bin apache2-data apache2-utils\n4 upgraded, 0 newly installed, 0 to remove and 44 not upgraded.\nNeed to get 1409 kB of archives.\nAfter this operation, 0 B of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 apache2 amd64 2.4.29-1ubuntu4.13 [95.1 kB]\nGet:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 apache2-bin amd64 2.4.29-1ubuntu4.13 [1070 kB]\nGet:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 apache2-utils amd64 2.4.29-1ubuntu4.13 [83.8 kB]\nGet:4 http://archive 2020-04-10T07:00:39.718 INFO:teuthology.task.ansible.out:.ubuntu.com/ubuntu bionic-updates/main amd64 apache2-data all 2.4.29-1ubuntu4.13 [160 kB]\nFetched 1409 kB in 1s (1665 kB/s)\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137332 files and directories currently installed.)\r\nPreparing to unpack .../apache2_2.4.29-1ubuntu4.13_amd64.deb ...\r\nUnpacking apache2 (2.4.29-1ubuntu4.13) over (2.4.29-1ubuntu4.12) ...\r\nPreparing to unpack .../apache2-bin_2.4.29-1ubuntu4.13_amd64.deb ...\r\nUnpacking apache2-bin (2.4.29-1ubuntu4.13) over (2.4.29-1ubuntu4.12) ...\r\nPreparing to unpack .../apache2-utils_2.4.29-1ubuntu4.13_amd64.deb ...\r\nUnpacking apache2-utils (2.4.29-1ubuntu4.13) over (2.4.29-1ubuntu4.12) ...\r\nPreparing to unpack .../apache2-data_2.4.29-1ubuntu4.13_all.deb ...\r\nUnpacking apache2-data (2.4.29-1ubuntu4.13) over (2.4.29-1ubuntu4.12) ...\r\nSetting up apache2-utils (2.4.29-1ubuntu4.13) ...\r\nSetting up apache2-bin (2.4.29-1ubuntu4.13) ...\r\nSetting up apache2-data (2.4.29-1ubuntu4.13) ...\r\nSetting up apache2 (2.4.29-1ubuntu4.13) ...\r\nProcessing triggers for systemd (237-3ubuntu10.39) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for ufw (0.36-0ubuntu0.18.04.1) ...\r\nProcessing triggers for ureadahead (0.100.0-21) ...\r\nureadahead will be reprofiled on next reboot\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed an 2020-04-10T07:00:39.727 INFO:teuthology.task.ansible.out:d are no longer required:", " cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good", " gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0", " libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1", " libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0", " libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0", " libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3", " libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0", " libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5", " libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following additional packages will be installed:", " apache2-bin apache2-data apache2-utils", "Suggested packages:", " www-browser apache2-doc apache2-suexec-pristine | apache2-suexec-custom", "The following packages will be upgraded:", " apache2 apache2-bin apache2-data apache2-utils", "4 upgraded, 0 newly installed, 0 to remove and 44 not upgraded.", "Need to get 1409 kB of archives.", "After this operation, 0 B of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 apache2 amd64 2.4.29-1ubuntu4.13 [95.1 kB]", "Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 apache2-bin amd64 2.4.29-1ubuntu4.13 [1070 kB]", "Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 apache2-utils amd64 2.4.29-1ubuntu4.13 [83.8 kB]", "Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 apache2-data all 2.4.29-1ubuntu4.13 [160 kB]", "Fetched 1409 kB in 1s (1665 kB/s)", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading dat 2020-04-10T07:00:39.734 INFO:teuthology.task.ansible.out:abase ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137332 files and directories currently installed.)", "Preparing to unpack .../apache2_2.4.29-1ubuntu4.13_amd64.deb ...", "Unpacking apache2 (2.4.29-1ubuntu4.13) over (2.4.29-1ubuntu4.12) ...", "Preparing to unpack .../apache2-bin_2.4.29-1ubuntu4.13_amd64.deb ...", "Unpacking apache2-bin (2.4.29-1ubuntu4.13) over (2.4.29-1ubuntu4.12) ...", "Preparing to unpack .../apache2-utils_2.4.29-1ubuntu4.13_amd64.deb ...", "Unpacking apache2-utils (2.4.29-1ubuntu4.13) over (2.4.29-1ubuntu4.12) ...", "Preparing to unpack .../apache2-data_2.4.29-1ubuntu4.13_all.deb ...", "Unpacking apache2-data (2.4.29-1ubuntu4.13) over (2.4.29-1ubuntu4.12) ...", "Setting up apache2-utils (2.4.29-1ubuntu4.13) ...", "Setting up apache2-bin (2.4.29-1ubuntu4.13) ...", "Setting up apache2-data (2.4.29-1ubuntu4.13) ...", "Setting up apache2 (2.4.29-1ubuntu4.13) ...", "Processing triggers for systemd (237-3ubuntu10.39) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for ufw (0.36-0ubuntu0.18.04.1) ...", "Processing triggers for ureadahead (0.100.0-21) ...", "ureadahead will be reprofiled on next reboot"]} 2020-04-10T07:00:39.766 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:39 +0000 (0:00:15.168) 0:01:50.219 ********** 2020-04-10T07:00:39.794 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2020-04-10T07:00:39.796 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:39.841 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:39 +0000 (0:00:00.075) 0:01:50.294 ********** 2020-04-10T07:00:40.038 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* changed: [smithi101.front.sepia.ceph.com] => {"cache_update_time": 1586501716, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good\n gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0\n libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1\n libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0\n libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0\n libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3\n libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0\n libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5\n libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following additional packages will be installed:\n apache2-bin apache2-data apache2-utils\nSuggested packages:\n www-browser apache2-doc apache2-suexec-pristine | apache2-suexec-custom\nThe following packages will be upgraded:\n apache2 apache2-bin apache2-data apache2-utils\n4 upgraded, 0 newly installed, 0 to remove and 44 not upgraded.\nNeed to get 1409 kB of archives.\nAfter this operation, 0 B of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 apache2 amd64 2.4.29-1ubuntu4.13 [95.1 kB]\nGet:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 apache2-bin amd64 2.4.29-1ubuntu4.13 [1070 kB]\nGet:3 http://archive.ubuntu.com/ubuntu bionic-updat 2020-04-10T07:00:40.047 INFO:teuthology.task.ansible.out:es/main amd64 apache2-utils amd64 2.4.29-1ubuntu4.13 [83.8 kB]\nGet:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 apache2-data all 2.4.29-1ubuntu4.13 [160 kB]\nFetched 1409 kB in 1s (1696 kB/s)\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137332 files and directories currently installed.)\r\nPreparing to unpack .../apache2_2.4.29-1ubuntu4.13_amd64.deb ...\r\nUnpacking apache2 (2.4.29-1ubuntu4.13) over (2.4.29-1ubuntu4.12) ...\r\nPreparing to unpack .../apache2-bin_2.4.29-1ubuntu4.13_amd64.deb ...\r\nUnpacking apache2-bin (2.4.29-1ubuntu4.13) over (2.4.29-1ubuntu4.12) ...\r\nPreparing to unpack .../apache2-utils_2.4.29-1ubuntu4.13_amd64.deb ...\r\nUnpacking apache2-utils (2.4.29-1ubuntu4.13) over (2.4.29-1ubuntu4.12) ...\r\nPreparing to unpack .../apache2-data_2.4.29-1ubuntu4.13_all.deb ...\r\nUnpacking apache2-data (2.4.29-1ubuntu4.13) over (2.4.29-1ubuntu4.12) ...\r\nSetting up apache2-utils (2.4.29-1ubuntu4.13) ...\r\nSetting up apache2-bin (2.4.29-1ubuntu4.13) ...\r\nSetting up apache2-data (2.4.29-1ubuntu4.13) ...\r\nSetting up apache2 (2.4.29-1ubuntu4.13) ...\r\nProcessing triggers for systemd (237-3ubuntu10.39) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for ufw (0.36-0ubuntu0.18.04.1) ...\r\nProcessing triggers for ureadahead (0.100.0-21) ...\r\nureadahead will be reprofiled on next reboot\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Re 2020-04-10T07:00:40.050 INFO:teuthology.task.ansible.out:ading state information...", "The following packages were automatically installed and are no longer required:", " cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good", " gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0", " libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1", " libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0", " libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0", " libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3", " libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0", " libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5", " libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following additional packages will be installed:", " apache2-bin apache2-data apache2-utils", "Suggested packages:", " www-browser apache2-doc apache2-suexec-pristine | apache2-suexec-custom", "The following packages will be upgraded:", " apache2 apache2-bin apache2-data apache2-utils", "4 upgraded, 0 newly installed, 0 to remove and 44 not upgraded.", "Need to get 1409 kB of archives.", "After this operation, 0 B of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 apache2 amd64 2.4.29-1ubuntu4.13 [95.1 kB]", "Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 apache2-bin amd64 2.4.29-1ubuntu4.13 [1070 kB]", "Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 apache2-utils amd64 2.4.29-1ubuntu4.13 [83.8 kB]", "Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 apache2-data all 2.4.29-1ubuntu4.13 [160 kB]", "Fetched 1409 kB in 1s (1696 kB/s)", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading databa 2020-04-10T07:00:40.052 INFO:teuthology.task.ansible.out:se ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137332 files and directories currently installed.)", "Preparing to unpack .../apache2_2.4.29-1ubuntu4.13_amd64.deb ...", "Unpacking apache2 (2.4.29-1ubuntu4.13) over (2.4.29-1ubuntu4.12) ...", "Preparing to unpack .../apache2-bin_2.4.29-1ubuntu4.13_amd64.deb ...", "Unpacking apache2-bin (2.4.29-1ubuntu4.13) over (2.4.29-1ubuntu4.12) ...", "Preparing to unpack .../apache2-utils_2.4.29-1ubuntu4.13_amd64.deb ...", "Unpacking apache2-utils (2.4.29-1ubuntu4.13) over (2.4.29-1ubuntu4.12) ...", "Preparing to unpack .../apache2-data_2.4.29-1ubuntu4.13_all.deb ...", "Unpacking apache2-data (2.4.29-1ubuntu4.13) over (2.4.29-1ubuntu4.12) ...", "Setting up apache2-utils (2.4.29-1ubuntu4.13) ...", "Setting up apache2-bin (2.4.29-1ubuntu4.13) ...", "Setting up apache2-data (2.4.29-1ubuntu4.13) ...", "Setting up apache2 (2.4.29-1ubuntu4.13) ...", "Processing triggers for systemd (237-3ubuntu10.39) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for ufw (0.36-0ubuntu0.18.04.1) ...", "Processing triggers for ureadahead (0.100.0-21) ...", "ureadahead will be reprofiled on next reboot"]} 2020-04-10T07:00:40.086 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:40 +0000 (0:00:00.244) 0:01:50.539 ********** 2020-04-10T07:00:40.125 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:40.167 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:40 +0000 (0:00:00.080) 0:01:50.619 ********** 2020-04-10T07:00:49.018 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-04-10T07:00:49.019 INFO:teuthology.task.ansible.out:changed: [smithi204.front.sepia.ceph.com] => {"cache_update_time": 1586501758, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\ngit is already the newest version (1:2.17.1-1ubuntu0.5).\nThe following additional packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils\nSuggested packages:\n samba vde2 sgabios ovmf debootstrap\nThe following NEW packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 6 newly installed, 0 to remove and 44 not upgraded.\nNeed to get 10.4 MB of archives.\nAfter this operation, 47.5 MB of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu bionic-updates/main am 2020-04-10T07:00:49.019 INFO:teuthology.task.ansible.out:d64 librados2 amd64 12.2.12-0ubuntu0.18.04.5 [2704 kB]\nGet:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.12-0ubuntu0.18.04.5 [922 kB]\nGet:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.23 [39.6 kB]\nGet:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.23 [672 kB]\nGet:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.23 [5202 kB]\nGet:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.23 [871 kB]\nFetched 10.4 MB in 1s (9639 kB/s)\nSelecting previously unselected package librados2.\r\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database .. 2020-04-10T07:00:49.021 INFO:teuthology.task.ansible.out:. 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137332 files and directories currently installed.)\r\nPreparing to unpack .../0-librados2_12.2.12-0ubuntu0.18.04.5_amd64.deb ...\r\nUnpacking librados2 (12.2.12-0ubuntu0.18.04.5) ...\r\nSelecting previously unselected package librbd1.\r\nPreparing to unpack .../1-librbd1_12.2.12-0ubuntu0.18.04.5_amd64.deb ...\r\nUnpacking librbd1 (12.2.12-0ubuntu0.18.04.5) ...\r\nSelecting previously unselected package qemu-block-extra:amd64.\r\nPreparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.23_amd64.deb ...\r\nUnpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.23) ...\r\nSelecting previously unselected package qemu-system-common.\r\nPreparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.23_amd64.deb ...\r\nUnpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.23) ...\r\nSelecting previously unselected package qemu-system-x86.\r\nPreparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.23_amd64.deb ...\r\nUnpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.23) ...\r\nSelecting previously unselected package qemu-utils.\r\nPreparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.23_amd64.deb ...\r\nUnpacking qemu-utils (1:2.11+dfsg-1ubuntu7.23) ...\r\nSetting up librados2 (12.2.12-0ubuntu0.18.04.5) ...\r\nSetting up librbd1 (12.2.12-0ubuntu0.18.04.5) ...\r\nSetting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.23) ...\r\nSetting up qemu-utils (1:2.11+dfsg-1ubuntu7.23) ...\r\nSetting up qemu-system-common (1:2.11+dfsg-1ubuntu7.23) ...\r\nSetting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.23) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubunt 2020-04-10T07:00:49.022 INFO:teuthology.task.ansible.out:u1) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "git is already the newest version (1:2.17.1-1ubuntu0.5).", "The following additional packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils", "Suggested packages:", " samba vde2 sgabios ovmf debootstrap", "The following NEW packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 6 newly installed, 0 to remove and 44 not upgraded.", "Need to get 10.4 MB of archives.", "After this operation, 47.5 MB of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.12-0ubuntu0.18.04.5 [2704 kB]", "Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.12-0ubuntu0.18.04.5 [922 kB]", "Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.23 [39.6 kB]", "Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.23 [672 kB]", "Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.23 [5202 kB]", "Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.23 [871 kB]", "Fetched 10.4 MB in 1s (9639 kB/s)", "Selecting previously unselected package librados2.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Rea 2020-04-10T07:00:49.024 INFO:teuthology.task.ansible.out:ding database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137332 files and directories currently installed.)", "Preparing to unpack .../0-librados2_12.2.12-0ubuntu0.18.04.5_amd64.deb ...", "Unpacking librados2 (12.2.12-0ubuntu0.18.04.5) ...", "Selecting previously unselected package librbd1.", "Preparing to unpack .../1-librbd1_12.2.12-0ubuntu0.18.04.5_amd64.deb ...", "Unpacking librbd1 (12.2.12-0ubuntu0.18.04.5) ...", "Selecting previously unselected package qemu-block-extra:amd64.", "Preparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.23_amd64.deb ...", "Unpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.23) ...", "Selecting previously unselected package qemu-system-common.", "Preparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.23_amd64.deb ...", "Unpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.23) ...", "Selecting previously unselected package qemu-system-x86.", "Preparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.23_amd64.deb ...", "Unpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.23) ...", "Selecting previously unselected package qemu-utils.", "Preparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.23_amd64.deb ...", "Unpacking qemu-utils (1:2.11+dfsg-1ubuntu7.23) ...", "Setting up librados2 (12.2.12-0ubuntu0.18.04.5) ...", "Setting up librbd1 (12.2.12-0ubuntu0.18.04.5) ...", "Setting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.23) ...", "Setting up qemu-utils (1:2.11+dfsg-1ubuntu7.23) ...", "Setting up qemu-system-common (1:2.11+dfsg-1ubuntu7.23) ...", "Setting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.23) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1) ..."]} 2020-04-10T07:00:49.059 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:49 +0000 (0:00:08.893) 0:01:59.512 ********** 2020-04-10T07:00:49.376 INFO:teuthology.task.ansible.out:changed: [smithi101.front.sepia.ceph.com] => {"cache_update_time": 1586501716, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\ngit is already the newest version (1:2.17.1-1ubuntu0.5).\nThe following additional packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils\nSuggested packages:\n samba vde2 sgabios ovmf debootstrap\nThe following NEW packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 6 newly installed, 0 to remove and 44 not upgraded.\nNeed to get 10.4 MB of archives.\nAfter this operation, 47.5 MB of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.12-0ubuntu0.18.04.5 [2704 kB]\nGet:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.12-0ubuntu0.18.04.5 [922 kB]\nGet:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.23 [39.6 kB]\nGet:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.23 [672 kB]\nGet:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.23 [5202 kB]\nGet:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.23 [871 kB]\nFetched 10.4 MB in 1s (9615 kB/s)\nSelecting previously unselected package librados2.\r\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database .. 2020-04-10T07:00:49.378 INFO:teuthology.task.ansible.out:. 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137332 files and directories currently installed.)\r\nPreparing to unpack .../0-librados2_12.2.12-0ubuntu0.18.04.5_amd64.deb ...\r\nUnpacking librados2 (12.2.12-0ubuntu0.18.04.5) ...\r\nSelecting previously unselected package librbd1.\r\nPreparing to unpack .../1-librbd1_12.2.12-0ubuntu0.18.04.5_amd64.deb ...\r\nUnpacking librbd1 (12.2.12-0ubuntu0.18.04.5) ...\r\nSelecting previously unselected package qemu-block-extra:amd64.\r\nPreparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.23_amd64.deb ...\r\nUnpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.23) ...\r\nSelecting previously unselected package qemu-system-common.\r\nPreparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.23_amd64.deb ...\r\nUnpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.23) ...\r\nSelecting previously unselected package qemu-system-x86.\r\nPreparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.23_amd64.deb ...\r\nUnpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.23) ...\r\nSelecting previously unselected package qemu-utils.\r\nPreparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.23_amd64.deb ...\r\nUnpacking qemu-utils (1:2.11+dfsg-1ubuntu7.23) ...\r\nSetting up librados2 (12.2.12-0ubuntu0.18.04.5) ...\r\nSetting up librbd1 (12.2.12-0ubuntu0.18.04.5) ...\r\nSetting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.23) ...\r\nSetting up qemu-utils (1:2.11+dfsg-1ubuntu7.23) ...\r\nSetting up qemu-system-common (1:2.11+dfsg-1ubuntu7.23) ...\r\nSetting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.23) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubunt 2020-04-10T07:00:49.379 INFO:teuthology.task.ansible.out:u1) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "git is already the newest version (1:2.17.1-1ubuntu0.5).", "The following additional packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils", "Suggested packages:", " samba vde2 sgabios ovmf debootstrap", "The following NEW packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 6 newly installed, 0 to remove and 44 not upgraded.", "Need to get 10.4 MB of archives.", "After this operation, 47.5 MB of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.12-0ubuntu0.18.04.5 [2704 kB]", "Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.12-0ubuntu0.18.04.5 [922 kB]", "Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.23 [39.6 kB]", "Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.23 [672 kB]", "Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.23 [5202 kB]", "Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.23 [871 kB]", "Fetched 10.4 MB in 1s (9615 kB/s)", "Selecting previously unselected package librados2.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Rea 2020-04-10T07:00:49.381 INFO:teuthology.task.ansible.out:ding database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137332 files and directories currently installed.)", "Preparing to unpack .../0-librados2_12.2.12-0ubuntu0.18.04.5_amd64.deb ...", "Unpacking librados2 (12.2.12-0ubuntu0.18.04.5) ...", "Selecting previously unselected package librbd1.", "Preparing to unpack .../1-librbd1_12.2.12-0ubuntu0.18.04.5_amd64.deb ...", "Unpacking librbd1 (12.2.12-0ubuntu0.18.04.5) ...", "Selecting previously unselected package qemu-block-extra:amd64.", "Preparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.23_amd64.deb ...", "Unpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.23) ...", "Selecting previously unselected package qemu-system-common.", "Preparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.23_amd64.deb ...", "Unpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.23) ...", "Selecting previously unselected package qemu-system-x86.", "Preparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.23_amd64.deb ...", "Unpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.23) ...", "Selecting previously unselected package qemu-utils.", "Preparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.23_amd64.deb ...", "Unpacking qemu-utils (1:2.11+dfsg-1ubuntu7.23) ...", "Setting up librados2 (12.2.12-0ubuntu0.18.04.5) ...", "Setting up librbd1 (12.2.12-0ubuntu0.18.04.5) ...", "Setting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.23) ...", "Setting up qemu-utils (1:2.11+dfsg-1ubuntu7.23) ...", "Setting up qemu-system-common (1:2.11+dfsg-1ubuntu7.23) ...", "Setting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.23) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1) ..."]} 2020-04-10T07:00:49.419 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:49 +0000 (0:00:00.359) 0:01:59.872 ********** 2020-04-10T07:00:49.907 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2020-04-10T07:00:49.908 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => {"cache_update_time": 1586501758, "cache_updated": false, "changed": false} 2020-04-10T07:00:49.948 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:49 +0000 (0:00:00.528) 0:02:00.401 ********** 2020-04-10T07:00:50.086 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2020-04-10T07:00:50.087 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => {"changed": false, "cmd": ["perl", "-pe", "if (m{^([^#]\\S*\\s+/\\s+\\S+\\s+)(\\S+)(\\s+.*)$}) { $_=\"$1$2,user_xattr$3\\n\" unless $2=~m{(^|,)user_xattr(,|$)}; }", "-i.bak", "/etc/fstab"], "rc": 0, "stdout": "skipped, since /etc/fstab.bak exists", "stdout_lines": ["skipped, since /etc/fstab.bak exists"]} 2020-04-10T07:00:50.160 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:50 +0000 (0:00:00.211) 0:02:00.613 ********** 2020-04-10T07:00:50.189 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** 2020-04-10T07:00:50.191 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:50.246 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:50 +0000 (0:00:00.085) 0:02:00.698 ********** 2020-04-10T07:00:50.276 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2020-04-10T07:00:50.277 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => {"cache_update_time": 1586501716, "cache_updated": false, "changed": false} 2020-04-10T07:00:50.331 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:50 +0000 (0:00:00.085) 0:02:00.784 ********** 2020-04-10T07:00:50.391 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ 2020-04-10T07:00:50.393 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => {"changed": false, "gid": 10102, "name": "fuse", "state": "present", "system": false} 2020-04-10T07:00:50.434 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:50 +0000 (0:00:00.102) 0:02:00.887 ********** 2020-04-10T07:00:50.462 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2020-04-10T07:00:50.462 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => {"changed": false, "cmd": ["perl", "-pe", "if (m{^([^#]\\S*\\s+/\\s+\\S+\\s+)(\\S+)(\\s+.*)$}) { $_=\"$1$2,user_xattr$3\\n\" unless $2=~m{(^|,)user_xattr(,|$)}; }", "-i.bak", "/etc/fstab"], "rc": 0, "stdout": "skipped, since /etc/fstab.bak exists", "stdout_lines": ["skipped, since /etc/fstab.bak exists"]} 2020-04-10T07:00:50.505 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:50 +0000 (0:00:00.071) 0:02:00.959 ********** 2020-04-10T07:00:50.532 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** 2020-04-10T07:00:50.533 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:50.574 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:50 +0000 (0:00:00.068) 0:02:01.027 ********** 2020-04-10T07:00:50.719 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ ok: [smithi101.front.sepia.ceph.com] => {"changed": false, "gid": 10102, "name": "fuse", "state": "present", "system": false} 2020-04-10T07:00:50.778 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:50 +0000 (0:00:00.203) 0:02:01.231 ********** 2020-04-10T07:00:50.795 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** 2020-04-10T07:00:50.796 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => {"changed": false, "checksum": "9edff47031d0f14a9b8c813b44eb4a9296d0439f", "dest": "/etc/fuse.conf", "gid": 10102, "group": "fuse", "mode": "0644", "owner": "root", "path": "/etc/fuse.conf", "size": 366, "state": "file", "uid": 0} 2020-04-10T07:00:50.841 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:50 +0000 (0:00:00.063) 0:02:01.294 ********** 2020-04-10T07:00:51.013 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ 2020-04-10T07:00:51.013 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1211, "groups": "fuse,kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2020-04-10T07:00:51.055 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:51 +0000 (0:00:00.213) 0:02:01.508 ********** 2020-04-10T07:00:51.117 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** 2020-04-10T07:00:51.119 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => {"changed": false, "checksum": "9edff47031d0f14a9b8c813b44eb4a9296d0439f", "dest": "/etc/fuse.conf", "gid": 10102, "group": "fuse", "mode": "0644", "owner": "root", "path": "/etc/fuse.conf", "size": 366, "state": "file", "uid": 0} 2020-04-10T07:00:51.181 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:51 +0000 (0:00:00.126) 0:02:01.635 ********** 2020-04-10T07:00:51.210 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** 2020-04-10T07:00:51.211 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-04-10T07:00:51.253 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:51 +0000 (0:00:00.071) 0:02:01.706 ********** 2020-04-10T07:00:51.362 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ 2020-04-10T07:00:51.369 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1211, "groups": "fuse,kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2020-04-10T07:00:51.415 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:51 +0000 (0:00:00.161) 0:02:01.868 ********** 2020-04-10T07:00:51.539 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2020-04-10T07:00:51.542 INFO:teuthology.task.ansible.out:changed: [smithi204.front.sepia.ceph.com] => {"changed": true, "name": "apache2", "state": "stopped", "status": {"ActiveEnterTimestamp": "Fri 2020-04-10 07:00:37 UTC", "ActiveEnterTimestampMonotonic": "137209621", "ActiveExitTimestamp": "Fri 2020-04-10 07:00:27 UTC", "ActiveExitTimestampMonotonic": "126645646", "ActiveState": "active", "After": "system.slice basic.target systemd-tmpfiles-setup.service network.target remote-fs.target nss-lookup.target sysinit.target systemd-journald.socket -.mount", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Fri 2020-04-10 07:00:37 UTC", "AssertTimestampMonotonic": "137185852", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": " 2020-04-10T07:00:51.543 INFO:teuthology.task.ansible.out:0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Fri 2020-04-10 07:00:37 UTC", "ConditionTimestampMonotonic": "137185852", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target", "ControlGroup": "/system.slice/apache2.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "The Apache HTTP Server", "DevicePolicy": "auto", "DropInPaths": "/lib/systemd/system/apache2.service.d/apache2-systemd.conf", "DynamicUser": "no", "Environment": "APACHE_STARTED_BY_SYSTEMD=true", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "6656", "ExecMainStartTimestamp": "Fri 2020-04-10 07:00:37 UTC", "ExecMainStartTimestampMonotonic": "137209599", "ExecMainStatus": "0", "ExecReload": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl graceful ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl start ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStop": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl stop ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/s 2020-04-10T07:00:51.544 INFO:teuthology.task.ansible.out:ystem/apache2.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "apache2.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestamp": "Fri 2020-04-10 07:00:27 UTC", "InactiveEnterTimestampMonotonic": "126712147", "InactiveExitTimestamp": "Fri 2020-04-10 07:00:37 UTC", "InactiveExitTimestampMonotonic": "137186601", "InvocationID": "e35d9ebfac1b4e0da583e2387b758f5d", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128031", "LimitNPROCSoft": "128031", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128031", "LimitSIGPENDINGSoft": "128031", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "6656", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": 2020-04-10T07:00:51.546 INFO:teuthology.task.ansible.out:"infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "apache2.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "-.mount system.slice sysinit.target", "RequiresMountsFor": "/var/tmp /tmp", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Fri 2020-04-10 07:00:37 UTC", "StateChangeTimestampMonotonic": "137209621", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "55", "TasksMax": "4915", "TimeoutStartUSec": "1mi 2020-04-10T07:00:51.547 INFO:teuthology.task.ansible.out:n 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Fri 2020-04-10 07:00:37 UTC", "WatchdogTimestampMonotonic": "137209619", "WatchdogUSec": "0"}} 2020-04-10T07:00:51.610 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:51 +0000 (0:00:00.195) 0:02:02.063 ********** 2020-04-10T07:00:51.616 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** 2020-04-10T07:00:51.617 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-04-10T07:00:51.665 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:51 +0000 (0:00:00.053) 0:02:02.117 ********** 2020-04-10T07:00:51.669 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2020-04-10T07:00:51.671 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:51.711 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:51 +0000 (0:00:00.047) 0:02:02.164 ********** 2020-04-10T07:00:51.738 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2020-04-10T07:00:51.739 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:51.778 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:51 +0000 (0:00:00.067) 0:02:02.231 ********** 2020-04-10T07:00:51.804 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2020-04-10T07:00:51.806 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:51.847 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:51 +0000 (0:00:00.068) 0:02:02.300 ********** 2020-04-10T07:00:51.873 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2020-04-10T07:00:51.875 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:51.918 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:51 +0000 (0:00:00.067) 0:02:02.368 ********** 2020-04-10T07:00:51.943 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2020-04-10T07:00:51.944 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:51.985 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:51 +0000 (0:00:00.070) 0:02:02.438 ********** 2020-04-10T07:00:51.991 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2020-04-10T07:00:51.994 INFO:teuthology.task.ansible.out:changed: [smithi101.front.sepia.ceph.com] => {"changed": true, "name": "apache2", "state": "stopped", "status": {"ActiveEnterTimestamp": "Fri 2020-04-10 07:00:37 UTC", "ActiveEnterTimestampMonotonic": "137028446", "ActiveExitTimestamp": "Fri 2020-04-10 07:00:27 UTC", "ActiveExitTimestampMonotonic": "126611814", "ActiveState": "active", "After": "network.target systemd-journald.socket systemd-tmpfiles-setup.service sysinit.target basic.target -.mount nss-lookup.target system.slice remote-fs.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Fri 2020-04-10 07:00:37 UTC", "AssertTimestampMonotonic": "136996455", "Before": "multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Fri 2020-04-10 07:00:37 UTC", "ConditionTimestampMonotonic": "136996455", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target", "ControlGroup": "/system.slice/apache2.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", " 2020-04-10T07:00:51.996 INFO:teuthology.task.ansible.out:Description": "The Apache HTTP Server", "DevicePolicy": "auto", "DropInPaths": "/lib/systemd/system/apache2.service.d/apache2-systemd.conf", "DynamicUser": "no", "Environment": "APACHE_STARTED_BY_SYSTEMD=true", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "6799", "ExecMainStartTimestamp": "Fri 2020-04-10 07:00:37 UTC", "ExecMainStartTimestampMonotonic": "137028422", "ExecMainStatus": "0", "ExecReload": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl graceful ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl start ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStop": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl stop ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/apache2.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "apache2.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestamp": "Fri 2020-04-10 07:00:27 UTC", "InactiveEnterTimestampMonotonic": "126688436", "InactiveExitTimestamp": "Fri 2020-04-10 07:00:37 UTC", "InactiveExitTimestampMonotonic": "136997453", "InvocationID": "bc616e1c21b947879231c29ddf0d4ece", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "i 2020-04-10T07:00:52.002 INFO:teuthology.task.ansible.out:nfinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128031", "LimitNPROCSoft": "128031", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128031", "LimitSIGPENDINGSoft": "128031", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "6799", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "apache2.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target -.mount system.slice", "RequiresMountsFor": "/var/tmp /tmp", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectory 2020-04-10T07:00:52.003 INFO:teuthology.task.ansible.out:Mode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Fri 2020-04-10 07:00:37 UTC", "StateChangeTimestampMonotonic": "137028446", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "55", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Fri 2020-04-10 07:00:37 UTC", "WatchdogTimestampMonotonic": "137028445", "WatchdogUSec": "0"}} 2020-04-10T07:00:52.040 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:52 +0000 (0:00:00.054) 0:02:02.493 ********** 2020-04-10T07:00:52.046 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2020-04-10T07:00:52.047 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:52.091 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:52 +0000 (0:00:00.048) 0:02:02.542 ********** 2020-04-10T07:00:52.097 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2020-04-10T07:00:52.097 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:52.142 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:52 +0000 (0:00:00.052) 0:02:02.594 ********** 2020-04-10T07:00:52.147 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2020-04-10T07:00:52.148 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:52.191 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:52 +0000 (0:00:00.050) 0:02:02.644 ********** 2020-04-10T07:00:52.197 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2020-04-10T07:00:52.198 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:52.241 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:52 +0000 (0:00:00.049) 0:02:02.694 ********** 2020-04-10T07:00:52.246 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2020-04-10T07:00:52.248 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:52.289 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:52 +0000 (0:00:00.048) 0:02:02.742 ********** 2020-04-10T07:00:52.296 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2020-04-10T07:00:52.297 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:52.341 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:52 +0000 (0:00:00.051) 0:02:02.793 ********** 2020-04-10T07:00:52.346 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-04-10T07:00:52.348 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:52.396 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:52 +0000 (0:00:00.055) 0:02:02.849 ********** 2020-04-10T07:00:52.403 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2020-04-10T07:00:52.408 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:52.451 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:52 +0000 (0:00:00.054) 0:02:02.904 ********** 2020-04-10T07:00:52.457 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2020-04-10T07:00:52.458 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:52.511 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:52 +0000 (0:00:00.059) 0:02:02.963 ********** 2020-04-10T07:00:52.515 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2020-04-10T07:00:52.517 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:52.560 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:52 +0000 (0:00:00.049) 0:02:03.013 ********** 2020-04-10T07:00:52.565 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-04-10T07:00:52.567 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:52.611 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:52 +0000 (0:00:00.051) 0:02:03.064 ********** 2020-04-10T07:00:52.619 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:52.663 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:52 +0000 (0:00:00.051) 0:02:03.116 ********** 2020-04-10T07:00:52.669 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-04-10T07:00:52.670 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:52.718 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:52 +0000 (0:00:00.055) 0:02:03.171 ********** 2020-04-10T07:00:52.723 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2020-04-10T07:00:52.726 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:52.768 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:52 +0000 (0:00:00.050) 0:02:03.222 ********** 2020-04-10T07:00:52.775 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2020-04-10T07:00:52.776 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:52.839 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:52 +0000 (0:00:00.062) 0:02:03.284 ********** TASK [testnode : Ensure ceph dependency packages are not present.] ************* skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:52.881 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:52 +0000 (0:00:00.049) 0:02:03.334 ********** 2020-04-10T07:00:52.966 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Friday 10 April 2020 07:00:52 +0000 (0:00:00.057) 0:02:03.392 ********** TASK [testnode : set_fact] ***************************************************** skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:52.986 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:52 +0000 (0:00:00.046) 0:02:03.439 ********** 2020-04-10T07:00:52.992 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-04-10T07:00:52.993 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:53.034 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:53 +0000 (0:00:00.048) 0:02:03.487 ********** 2020-04-10T07:00:53.040 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2020-04-10T07:00:53.040 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:53.089 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:53 +0000 (0:00:00.048) 0:02:03.536 ********** TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-04-10T07:00:53.090 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:53.133 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:53 +0000 (0:00:00.049) 0:02:03.586 ********** 2020-04-10T07:00:53.138 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-04-10T07:00:53.139 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:53.179 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:53 +0000 (0:00:00.046) 0:02:03.632 ********** 2020-04-10T07:00:53.186 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:53.227 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:53 +0000 (0:00:00.047) 0:02:03.680 ********** 2020-04-10T07:00:53.233 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-04-10T07:00:53.234 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:53.275 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:53 +0000 (0:00:00.047) 0:02:03.728 ********** 2020-04-10T07:00:53.280 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2020-04-10T07:00:53.282 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:53.327 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:53 +0000 (0:00:00.051) 0:02:03.780 ********** 2020-04-10T07:00:53.333 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2020-04-10T07:00:53.333 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:53.382 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:53 +0000 (0:00:00.055) 0:02:03.835 ********** 2020-04-10T07:00:53.389 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2020-04-10T07:00:53.390 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:53.437 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:53 +0000 (0:00:00.055) 0:02:03.890 ********** 2020-04-10T07:00:53.444 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-04-10T07:00:53.446 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:53.489 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:53 +0000 (0:00:00.051) 0:02:03.942 ********** 2020-04-10T07:00:53.496 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-04-10T07:00:53.497 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:53.545 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:53 +0000 (0:00:00.055) 0:02:03.998 ********** 2020-04-10T07:00:53.552 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-04-10T07:00:53.553 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:53.601 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:53 +0000 (0:00:00.056) 0:02:04.054 ********** 2020-04-10T07:00:53.609 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-04-10T07:00:53.610 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:53.651 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:53 +0000 (0:00:00.049) 0:02:04.104 ********** 2020-04-10T07:00:53.660 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:53.699 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:53 +0000 (0:00:00.047) 0:02:04.151 ********** 2020-04-10T07:00:53.705 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-04-10T07:00:53.706 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:53.753 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:53 +0000 (0:00:00.055) 0:02:04.207 ********** 2020-04-10T07:00:53.760 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2020-04-10T07:00:53.761 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:53.806 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:53 +0000 (0:00:00.052) 0:02:04.259 ********** 2020-04-10T07:00:53.811 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-04-10T07:00:53.812 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:53.855 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:53 +0000 (0:00:00.049) 0:02:04.308 ********** 2020-04-10T07:00:53.861 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:53.910 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:53 +0000 (0:00:00.054) 0:02:04.362 ********** 2020-04-10T07:00:53.916 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-04-10T07:00:53.916 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:53.958 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:53 +0000 (0:00:00.049) 0:02:04.411 ********** 2020-04-10T07:00:53.964 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2020-04-10T07:00:53.965 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:54.005 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:54 +0000 (0:00:00.046) 0:02:04.458 ********** 2020-04-10T07:00:54.011 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-04-10T07:00:54.012 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:54.057 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:54 +0000 (0:00:00.051) 0:02:04.510 ********** 2020-04-10T07:00:54.064 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:54.106 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:54 +0000 (0:00:00.049) 0:02:04.559 ********** 2020-04-10T07:00:54.113 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* 2020-04-10T07:00:54.114 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:54.159 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:54 +0000 (0:00:00.051) 0:02:04.611 ********** 2020-04-10T07:00:54.164 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-04-10T07:00:54.165 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:54.208 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:54 +0000 (0:00:00.049) 0:02:04.661 ********** 2020-04-10T07:00:54.236 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:54.277 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:54 +0000 (0:00:00.069) 0:02:04.730 ********** 2020-04-10T07:00:54.303 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-04-10T07:00:54.305 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:54.347 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:54 +0000 (0:00:00.070) 0:02:04.800 ********** 2020-04-10T07:00:54.354 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2020-04-10T07:00:54.361 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => {"changed": false, "path": "/etc/ceph", "state": "absent"} 2020-04-10T07:00:54.397 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:54 +0000 (0:00:00.049) 0:02:04.850 ********** 2020-04-10T07:00:54.403 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-04-10T07:00:54.403 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:54.447 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:54 +0000 (0:00:00.050) 0:02:04.900 ********** 2020-04-10T07:00:54.478 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-04-10T07:00:54.480 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:54.518 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:54 +0000 (0:00:00.071) 0:02:04.971 ********** 2020-04-10T07:00:54.545 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* 2020-04-10T07:00:54.547 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:54.587 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:54 +0000 (0:00:00.068) 0:02:05.040 ********** 2020-04-10T07:00:54.722 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2020-04-10T07:00:54.724 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => {"changed": false, "path": "/etc/ceph", "state": "absent"} 2020-04-10T07:00:54.769 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:54 +0000 (0:00:00.181) 0:02:05.222 ********** 2020-04-10T07:00:54.775 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2020-04-10T07:00:54.776 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => {"changed": false, "checksum": "0b949a4622a315acfc90571fddbd013b4dc0f5f9", "dest": "/etc/exports", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/exports", "size": 578, "state": "file", "uid": 0} 2020-04-10T07:00:54.817 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:54 +0000 (0:00:00.047) 0:02:05.270 ********** 2020-04-10T07:00:54.844 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2020-04-10T07:00:54.845 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:54.885 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:54 +0000 (0:00:00.068) 0:02:05.338 ********** 2020-04-10T07:00:55.101 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2020-04-10T07:00:55.101 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => {"changed": false, "checksum": "0b949a4622a315acfc90571fddbd013b4dc0f5f9", "dest": "/etc/exports", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/exports", "size": 578, "state": "file", "uid": 0} 2020-04-10T07:00:55.142 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:55 +0000 (0:00:00.256) 0:02:05.595 ********** 2020-04-10T07:00:55.169 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2020-04-10T07:00:55.171 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:55.217 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:55 +0000 (0:00:00.074) 0:02:05.670 ********** 2020-04-10T07:00:55.222 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** 2020-04-10T07:00:55.223 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => {"changed": false, "checksum": "d5677bbd13857966eb0fc86f66195b18a2b98e20", "dest": "/etc/cron.weekly/kernel-clean", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/cron.weekly/kernel-clean", "size": 752, "state": "file", "uid": 0} 2020-04-10T07:00:55.269 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:55 +0000 (0:00:00.052) 0:02:05.722 ********** 2020-04-10T07:00:55.572 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => {"changed": false, "checksum": "d5677bbd13857966eb0fc86f66195b18a2b98e20", "dest": "/etc/cron.weekly/kernel-clean", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/cron.weekly/kernel-clean", "size": 752, "state": "file", "uid": 0} 2020-04-10T07:00:55.613 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:55 +0000 (0:00:00.344) 0:02:06.066 ********** 2020-04-10T07:00:55.618 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2020-04-10T07:00:55.619 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => {"changed": false, "checksum": "22e9a1446f43d0440219b9af14a66dd77fb0c2eb", "dest": "/etc/grub.d/02_force_timeout", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/grub.d/02_force_timeout", "size": 115, "state": "file", "uid": 0} 2020-04-10T07:00:55.661 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:55 +0000 (0:00:00.047) 0:02:06.114 ********** 2020-04-10T07:00:55.965 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => {"changed": false, "checksum": "22e9a1446f43d0440219b9af14a66dd77fb0c2eb", "dest": "/etc/grub.d/02_force_timeout", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/grub.d/02_force_timeout", "size": 115, "state": "file", "uid": 0} 2020-04-10T07:00:56.007 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:56 +0000 (0:00:00.346) 0:02:06.460 ********** 2020-04-10T07:00:56.026 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** 2020-04-10T07:00:56.028 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => {"changed": false, "checksum": "6449466d762bced58744df80432d189e35329803", "dest": "/etc/modules", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/modules", "size": 315, "state": "file", "uid": 0} 2020-04-10T07:00:56.076 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:56 +0000 (0:00:00.065) 0:02:06.526 ********** 2020-04-10T07:00:56.209 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2020-04-10T07:00:56.210 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-04-10T07:00:56.251 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:56 +0000 (0:00:00.177) 0:02:06.704 ********** 2020-04-10T07:00:56.279 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** 2020-04-10T07:00:56.280 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:56.319 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:56 +0000 (0:00:00.068) 0:02:06.773 ********** 2020-04-10T07:00:56.346 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ 2020-04-10T07:00:56.348 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:56.395 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:56 +0000 (0:00:00.075) 0:02:06.849 ********** 2020-04-10T07:00:56.402 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** 2020-04-10T07:00:56.403 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => {"changed": false, "checksum": "6449466d762bced58744df80432d189e35329803", "dest": "/etc/modules", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/modules", "size": 315, "state": "file", "uid": 0} 2020-04-10T07:00:56.457 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:56 +0000 (0:00:00.053) 0:02:06.903 ********** TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:56.499 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:56 +0000 (0:00:00.048) 0:02:06.951 ********** 2020-04-10T07:00:56.525 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:56.568 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:56 +0000 (0:00:00.069) 0:02:07.021 ********** 2020-04-10T07:00:56.587 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** ok: [smithi101.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-04-10T07:00:56.625 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:56 +0000 (0:00:00.057) 0:02:07.078 ********** 2020-04-10T07:00:56.632 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* 2020-04-10T07:00:56.633 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:56.673 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:56 +0000 (0:00:00.047) 0:02:07.126 ********** 2020-04-10T07:00:56.680 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:56.721 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:56 +0000 (0:00:00.047) 0:02:07.174 ********** 2020-04-10T07:00:56.727 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-04-10T07:00:56.728 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2020-04-10T07:00:56.768 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:56 +0000 (0:00:00.047) 0:02:07.222 ********** 2020-04-10T07:00:56.775 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ 2020-04-10T07:00:56.776 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:56.823 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:56 +0000 (0:00:00.054) 0:02:07.276 ********** 2020-04-10T07:00:56.830 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2020-04-10T07:00:56.831 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2020-04-10T07:00:56.843 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item=loop3) => {"ansible_loop_var": "item", "changed": false, "item": "loop3", "skip_reason": "Conditional result was False"} 2020-04-10T07:00:56.845 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item=loop2) => {"ansible_loop_var": "item", "changed": false, "item": "loop2", "skip_reason": "Conditional result was False"} skipping: [smithi204.front.sepia.ceph.com] => (item=loop1) => {"ansible_loop_var": "item", "changed": false, "item": "loop1", "skip_reason": "Conditional result was False"} 2020-04-10T07:00:56.855 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* 2020-04-10T07:00:56.856 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check for NVMe drive] ***************************************** 2020-04-10T07:00:56.857 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item=loop0) => {"ansible_loop_var": "item", "changed": false, "item": "loop0", "skip_reason": "Conditional result was False"} 2020-04-10T07:00:56.912 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:56 +0000 (0:00:00.089) 0:02:07.365 ********** 2020-04-10T07:00:56.920 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item=loop7) => {"ansible_loop_var": "item", "changed": false, "item": "loop7", "skip_reason": "Conditional result was False"} 2020-04-10T07:00:56.921 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item=loop6) => {"ansible_loop_var": "item", "changed": false, "item": "loop6", "skip_reason": "Conditional result was False"} 2020-04-10T07:00:56.922 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item=loop5) => {"ansible_loop_var": "item", "changed": false, "item": "loop5", "skip_reason": "Conditional result was False"} skipping: [smithi204.front.sepia.ceph.com] => (item=loop4) => {"ansible_loop_var": "item", "changed": false, "item": "loop4", "skip_reason": "Conditional result was False"} 2020-04-10T07:00:56.922 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2020-04-10T07:00:56.937 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2020-04-10T07:00:56.938 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2020-04-10T07:00:56.939 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:56.993 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:56 +0000 (0:00:00.076) 0:02:07.442 ********** 2020-04-10T07:00:56.996 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2020-04-10T07:00:56.996 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2020-04-10T07:00:56.997 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} skipping: [smithi204.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2020-04-10T07:00:57.049 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:57 +0000 (0:00:00.058) 0:02:07.502 ********** 2020-04-10T07:00:57.055 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* 2020-04-10T07:00:57.056 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:57.096 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:57 +0000 (0:00:00.047) 0:02:07.549 ********** 2020-04-10T07:00:57.133 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** ok: [smithi101.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2020-04-10T07:00:57.177 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:57 +0000 (0:00:00.080) 0:02:07.630 ********** 2020-04-10T07:00:57.214 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** 2020-04-10T07:00:57.215 INFO:teuthology.task.ansible.out:changed: [smithi204.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.004301", "end": "2020-04-10 07:00:57.187679", "rc": 0, "start": "2020-04-10 07:00:57.183378", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} 2020-04-10T07:00:57.286 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:57 +0000 (0:00:00.109) 0:02:07.739 ********** 2020-04-10T07:00:57.294 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2020-04-10T07:00:57.295 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2020-04-10T07:00:57.296 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item=loop3) => {"ansible_loop_var": "item", "changed": false, "item": "loop3", "skip_reason": "Conditional result was False"} 2020-04-10T07:00:57.298 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item=loop2) => {"ansible_loop_var": "item", "changed": false, "item": "loop2", "skip_reason": "Conditional result was False"} 2020-04-10T07:00:57.299 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item=loop1) => {"ansible_loop_var": "item", "changed": false, "item": "loop1", "skip_reason": "Conditional result was False"} 2020-04-10T07:00:57.301 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item=loop0) => {"ansible_loop_var": "item", "changed": false, "item": "loop0", "skip_reason": "Conditional result was False"} skipping: [smithi101.front.sepia.ceph.com] => (item=loop7) => {"ansible_loop_var": "item", "changed": false, "item": "loop7", "skip_reason": "Conditional result was False"} 2020-04-10T07:00:57.302 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item=loop6) => {"ansible_loop_var": "item", "changed": false, "item": "loop6", "skip_reason": "Conditional result was False"} 2020-04-10T07:00:57.305 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item=loop5) => {"ansible_loop_var": "item", "changed": false, "item": "loop5", "skip_reason": "Conditional result was False"} skipping: [smithi101.front.sepia.ceph.com] => (item=loop4) => {"ansible_loop_var": "item", "changed": false, "item": "loop4", "skip_reason": "Conditional result was False"} 2020-04-10T07:00:57.317 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2020-04-10T07:00:57.318 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2020-04-10T07:00:57.329 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ 2020-04-10T07:00:57.329 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:57.330 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2020-04-10T07:00:57.330 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2020-04-10T07:00:57.379 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:57 +0000 (0:00:00.093) 0:02:07.833 ********** 2020-04-10T07:00:57.389 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} skipping: [smithi101.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} skipping: [smithi101.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2020-04-10T07:00:57.446 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:57 +0000 (0:00:00.066) 0:02:07.899 ********** 2020-04-10T07:00:57.452 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** 2020-04-10T07:00:57.454 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:57.512 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:57 +0000 (0:00:00.065) 0:02:07.965 ********** 2020-04-10T07:00:57.553 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:57.604 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:57 +0000 (0:00:00.092) 0:02:08.057 ********** 2020-04-10T07:00:57.611 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** 2020-04-10T07:00:57.612 INFO:teuthology.task.ansible.out:changed: [smithi101.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.009548", "end": "2020-04-10 07:00:57.567354", "rc": 0, "start": "2020-04-10 07:00:57.557806", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} 2020-04-10T07:00:57.662 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:57 +0000 (0:00:00.057) 0:02:08.115 ********** 2020-04-10T07:00:57.700 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ 2020-04-10T07:00:57.701 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:57.743 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:57 +0000 (0:00:00.080) 0:02:08.196 ********** 2020-04-10T07:00:57.779 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** 2020-04-10T07:00:57.781 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:57.823 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:57 +0000 (0:00:00.080) 0:02:08.276 ********** 2020-04-10T07:00:57.860 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** 2020-04-10T07:00:57.862 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:57.905 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:57 +0000 (0:00:00.081) 0:02:08.358 ********** 2020-04-10T07:00:58.584 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2020-04-10T07:00:58.585 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => {"cache_update_time": 1586501758, "cache_updated": false, "changed": false} 2020-04-10T07:00:58.626 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:58 +0000 (0:00:00.721) 0:02:09.079 ********** 2020-04-10T07:00:58.662 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:58.709 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:58 +0000 (0:00:00.080) 0:02:09.160 ********** 2020-04-10T07:00:58.754 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-04-10T07:00:58.756 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item={u'block_used': 13740116, u'uuid': u'298ac6f2-5334-4198-9d30-0049606b867a', u'size_total': 983429562368, u'block_total': 240095108, u'mount': u'/', u'block_available': 226354992, u'size_available': 927150047232, u'fstype': u'ext4', u'inode_total': 61054976, u'inode_available': 60905745, u'device': u'/dev/sda1', u'inode_used': 149231, u'block_size': 4096, u'options': u'rw,relatime,errors=remount-ro,data=ordered'}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 226354992, "block_size": 4096, "block_total": 240095108, "block_used": 13740116, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905745, "inode_total": 61054976, "inode_used": 149231, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927150047232, "size_total": 983429562368, "uuid": "298ac6f2-5334-4198-9d30-0049606b867a"}} 2020-04-10T07:00:58.799 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:58 +0000 (0:00:00.092) 0:02:09.252 ********** 2020-04-10T07:00:58.805 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2020-04-10T07:00:58.806 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => {"cache_update_time": 1586501716, "cache_updated": false, "changed": false} 2020-04-10T07:00:58.849 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:58 +0000 (0:00:00.049) 0:02:09.302 ********** 2020-04-10T07:00:58.877 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2020-04-10T07:00:58.879 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:58.951 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:58 +0000 (0:00:00.097) 0:02:09.400 ********** 2020-04-10T07:00:58.953 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2020-04-10T07:00:58.954 INFO:teuthology.task.ansible.out:changed: [smithi204.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.009620", "end": "2020-04-10 07:00:59.114342", "rc": 0, "start": "2020-04-10 07:00:59.104722", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-04-10T07:00:58.998 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:58 +0000 (0:00:00.051) 0:02:09.451 ********** 2020-04-10T07:00:59.004 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-04-10T07:00:59.005 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item={u'block_used': 13740115, u'uuid': u'298ac6f2-5334-4198-9d30-0049606b867a', u'size_total': 983429562368, u'block_total': 240095108, u'mount': u'/', u'block_available': 226354993, u'size_available': 927150051328, u'fstype': u'ext4', u'inode_total': 61054976, u'inode_available': 60905745, u'device': u'/dev/sda1', u'inode_used': 149231, u'block_size': 4096, u'options': u'rw,relatime,errors=remount-ro,data=ordered'}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 226354993, "block_size": 4096, "block_total": 240095108, "block_used": 13740115, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905745, "inode_total": 61054976, "inode_used": 149231, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927150051328, "size_total": 983429562368, "uuid": "298ac6f2-5334-4198-9d30-0049606b867a"}} 2020-04-10T07:00:59.051 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:59 +0000 (0:00:00.052) 0:02:09.504 ********** 2020-04-10T07:00:59.057 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2020-04-10T07:00:59.058 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'block_used': 13740116, u'uuid': u'298ac6f2-5334-4198-9d30-0049606b867a', u'size_total': 983429562368, u'block_total': 240095108, u'mount': u'/', u'block_available': 226354992, u'size_available': 927150047232, u'fstype': u'ext4', u'inode_total': 61054976, u'inode_available': 60905745, u'device': u'/dev/sda1', u'inode_used': 149231, u'block_size': 4096, u'options': u'rw,relatime,errors=remount-ro,data=ordered'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226354992, "block_size": 4096, "block_total": 240095108, "block_used": 13740116, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905745, "inode_total": 61054976, "inode_used": 149231, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927150047232, "size_total": 983429562368, "uuid": "298ac6f2-5334-4198-9d30-0049606b867a"}, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:59.104 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:59 +0000 (0:00:00.052) 0:02:09.557 ********** 2020-04-10T07:00:59.133 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2020-04-10T07:00:59.174 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:59 +0000 (0:00:00.069) 0:02:09.627 ********** 2020-04-10T07:00:59.190 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2020-04-10T07:00:59.191 INFO:teuthology.task.ansible.out:changed: [smithi101.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.009877", "end": "2020-04-10 07:00:59.170761", "rc": 0, "start": "2020-04-10 07:00:59.160884", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-04-10T07:00:59.239 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:59 +0000 (0:00:00.064) 0:02:09.692 ********** 2020-04-10T07:00:59.247 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-04-10T07:00:59.248 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': u'ATA', u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W5XP2E', u'wwn-0x5000c50092941a36'], u'uuids': []}, u'sas_device_handle': None, u'host': u'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', u'support_discard': u'0', u'serial': u'Z1W5XP2E', u'wwn': u'0x5000c50092941a36', u'holders': [], u'size': u'931.51 GB', u'scheduler_mode': u'cfq', u'rotational': u'1', u'sectors': u'1953525168', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'ST1000NM0033-9ZM', u'partitions': {u'sda1': {u'start': u'2048', u'sectorsize': 512, u'uuid': u'298ac6f2-5334-4198-9d30-0049606b867a', u'sectors': u'1953522688', u'holders': [], u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W5XP2E-part1', u'wwn-0x5000c50092941a36-part1'], u'uuids': [u'298ac6f2-5334-4198-9d30-0049606b867a']}, u'size': u'931.51 GB'}}}, 'key': u'sda'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XP2E", "wwn-0x5000c50092941a36"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XP2E-part1", "wwn-0x5000c50092941a36-part1"], "labels": [], "masters": [], "uuids": ["298ac6f2-5334-4198-9d30-0049606b867a"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "298ac6f2-5334-4198-9d30-0049606b867a"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5XP2E", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "vir 2020-04-10T07:00:59.249 INFO:teuthology.task.ansible.out:tual": 1, "wwn": "0x5000c50092941a36"}}, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:59.281 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'0', u'model': None, u'size': u'0.00 Bytes', u'scheduler_mode': u'none', u'rotational': u'1', u'sectors': u'0', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'loop3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:59.283 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'0', u'model': None, u'size': u'0.00 Bytes', u'scheduler_mode': u'none', u'rotational': u'1', u'sectors': u'0', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'loop2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:59.295 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'0', u'model': None, u'size': u'0.00 Bytes', u'scheduler_mode': u'none', u'rotational': u'1', u'sectors': u'0', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'loop1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:59.310 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2020-04-10T07:00:59.311 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'block_used': 13740115, u'uuid': u'298ac6f2-5334-4198-9d30-0049606b867a', u'size_total': 983429562368, u'block_total': 240095108, u'mount': u'/', u'block_available': 226354993, u'size_available': 927150051328, u'fstype': u'ext4', u'inode_total': 61054976, u'inode_available': 60905745, u'device': u'/dev/sda1', u'inode_used': 149231, u'block_size': 4096, u'options': u'rw,relatime,errors=remount-ro,data=ordered'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226354993, "block_size": 4096, "block_total": 240095108, "block_used": 13740115, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905745, "inode_total": 61054976, "inode_used": 149231, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927150051328, "size_total": 983429562368, "uuid": "298ac6f2-5334-4198-9d30-0049606b867a"}, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:59.322 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-04-10T07:00:59.322 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'0', u'model': None, u'size': u'0.00 Bytes', u'scheduler_mode': u'none', u'rotational': u'1', u'sectors': u'0', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'loop0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:59.324 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'0', u'model': None, u'size': u'0.00 Bytes', u'scheduler_mode': u'none', u'rotational': u'1', u'sectors': u'0', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'loop7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:59.325 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2020-04-10T07:00:59.325 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-04-10T07:00:59.327 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'0', u'model': None, u'size': u'0.00 Bytes', u'scheduler_mode': u'none', u'rotational': u'1', u'sectors': u'0', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'loop6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:59.384 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:59 +0000 (0:00:00.144) 0:02:09.837 ********** 2020-04-10T07:00:59.392 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'0', u'model': None, u'size': u'0.00 Bytes', u'scheduler_mode': u'none', u'rotational': u'1', u'sectors': u'0', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'loop5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:59.394 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'0', u'model': None, u'size': u'0.00 Bytes', u'scheduler_mode': u'none', u'rotational': u'1', u'sectors': u'0', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'loop4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:59.408 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2020-04-10T07:00:59.453 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:00:59 +0000 (0:00:00.067) 0:02:09.904 ********** 2020-04-10T07:00:59.500 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-04-10T07:00:59.501 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': u'ATA', u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W5QCGY', u'wwn-0x5000c50091e05c18'], u'uuids': []}, u'sas_device_handle': None, u'host': u'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', u'support_discard': u'0', u'serial': u'Z1W5QCGY', u'wwn': u'0x5000c50091e05c18', u'holders': [], u'size': u'931.51 GB', u'scheduler_mode': u'cfq', u'rotational': u'1', u'sectors': u'1953525168', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'ST1000NM0033-9ZM', u'partitions': {u'sda1': {u'start': u'2048', u'sectorsize': 512, u'uuid': u'298ac6f2-5334-4198-9d30-0049606b867a', u'sectors': u'1953522688', u'holders': [], u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W5QCGY-part1', u'wwn-0x5000c50091e05c18-part1'], u'uuids': [u'298ac6f2-5334-4198-9d30-0049606b867a']}, u'size': u'931.51 GB'}}}, 'key': u'sda'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QCGY", "wwn-0x5000c50091e05c18"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QCGY-part1", "wwn-0x5000c50091e05c18-part1"], "labels": [], "masters": [], "uuids": ["298ac6f2-5334-4198-9d30-0049606b867a"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "298ac6f2-5334-4198-9d30-0049606b867a"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5QCGY", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "vir 2020-04-10T07:00:59.503 INFO:teuthology.task.ansible.out:tual": 1, "wwn": "0x5000c50091e05c18"}}, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:59.515 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'0', u'model': None, u'size': u'0.00 Bytes', u'scheduler_mode': u'none', u'rotational': u'1', u'sectors': u'0', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'loop3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:59.527 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'0', u'model': None, u'size': u'0.00 Bytes', u'scheduler_mode': u'none', u'rotational': u'1', u'sectors': u'0', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'loop2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:59.539 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'0', u'model': None, u'size': u'0.00 Bytes', u'scheduler_mode': u'none', u'rotational': u'1', u'sectors': u'0', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'loop1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:59.558 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'0', u'model': None, u'size': u'0.00 Bytes', u'scheduler_mode': u'none', u'rotational': u'1', u'sectors': u'0', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'loop0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:59.565 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'0', u'model': None, u'size': u'0.00 Bytes', u'scheduler_mode': u'none', u'rotational': u'1', u'sectors': u'0', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'loop7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:59.577 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'0', u'model': None, u'size': u'0.00 Bytes', u'scheduler_mode': u'none', u'rotational': u'1', u'sectors': u'0', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'loop6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:59.578 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'0', u'model': None, u'size': u'0.00 Bytes', u'scheduler_mode': u'none', u'rotational': u'1', u'sectors': u'0', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'loop5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-04-10T07:00:59.601 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'0', u'model': None, u'size': u'0.00 Bytes', u'scheduler_mode': u'none', u'rotational': u'1', u'sectors': u'0', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'loop4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:01.238 INFO:teuthology.task.ansible.out:changed: [smithi204.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [u'dm-0', u'dm-1', u'dm-2', u'dm-3', u'dm-4'], u'labels': [], u'ids': [u'lvm-pv-uuid-xQ1879-oYTb-0ddJ-eCMO-Cc4n-UdsM-U29Hj5', u'nvme-INTEL_SSDPEDMD400G4_CVFT623300GD400BGN', u'nvme-nvme.8086-43564654363233333030474434303042474e-494e54454c205353445045444d443430304734-00000001'], u'uuids': []}, u'sas_device_handle': None, u'host': u'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', u'support_discard': u'512', u'model': u'INTEL SSDPEDMD400G4', u'size': u'372.61 GB', u'scheduler_mode': u'none', u'rotational': u'0', u'sectors': u'781422768', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [u'vg_nvme-lv_4', u'vg_nvme-lv_1', u'vg_nvme-lv_3', u'vg_nvme-lv_5', u'vg_nvme-lv_2'], u'partitions': {}}, 'key': u'nvme0n1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.747595", "end": "2020-04-10 07:01:01.401277", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_4", "vg_nvme-lv_1", "vg_nvme-lv_3", "vg_nvme-lv_5", "vg_nvme-lv_2"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-xQ1879-oYTb-0ddJ-eCMO-Cc4n-UdsM-U29Hj5", "nvme-INTEL_SSDPEDMD400G4_CVFT623300GD400BGN", "nvme-nvme.8086-43564654363233333030474434303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-04-10 07:00:59.653682", "stderr": "", "stderr_lines": [], "stdout": "Creating new GP 2020-04-10T07:01:01.240 INFO:teuthology.task.ansible.out:T entries.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-04-10T07:01:01.429 INFO:teuthology.task.ansible.out:changed: [smithi101.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [u'dm-0', u'dm-1', u'dm-2', u'dm-3', u'dm-4'], u'labels': [], u'ids': [u'lvm-pv-uuid-AWWcD4-GqJ9-WTUT-MhbF-0NfY-1fdC-21akW9', u'nvme-INTEL_SSDPEDMD400G4_PHFT6204017B400BGN', u'nvme-nvme.8086-50484654363230343031374234303042474e-494e54454c205353445045444d443430304734-00000001'], u'uuids': []}, u'sas_device_handle': None, u'host': u'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', u'support_discard': u'512', u'model': u'INTEL SSDPEDMD400G4', u'size': u'372.61 GB', u'scheduler_mode': u'none', u'rotational': u'0', u'sectors': u'781422768', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [u'vg_nvme-lv_4', u'vg_nvme-lv_1', u'vg_nvme-lv_3', u'vg_nvme-lv_5', u'vg_nvme-lv_2'], u'partitions': {}}, 'key': u'nvme0n1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.710009", "end": "2020-04-10 07:01:01.404974", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_4", "vg_nvme-lv_1", "vg_nvme-lv_3", "vg_nvme-lv_5", "vg_nvme-lv_2"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-AWWcD4-GqJ9-WTUT-MhbF-0NfY-1fdC-21akW9", "nvme-INTEL_SSDPEDMD400G4_PHFT6204017B400BGN", "nvme-nvme.8086-50484654363230343031374234303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-04-10 07:00:59.694965", "stderr": "", "stderr_lines": [], "stdout": "Creating new GP 2020-04-10T07:01:01.430 INFO:teuthology.task.ansible.out:T entries.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-04-10T07:01:02.522 INFO:teuthology.task.ansible.out:changed: [smithi204.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_1', u'dm-uuid-LVM-qYMUetxG7lg5cwi90Cl7E9dn0vkL495ycD1SQr1DrycguuuXSTFBH6fnExcFcRcA'], u'uuids': [u'9601b024-cb25-4d3f-9528-e5def0570c6f']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-4'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.169389", "end": "2020-04-10 07:01:02.692919", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-qYMUetxG7lg5cwi90Cl7E9dn0vkL495ycD1SQr1DrycguuuXSTFBH6fnExcFcRcA"], "labels": [], "masters": [], "uuids": ["9601b024-cb25-4d3f-9528-e5def0570c6f"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-04-10 07:01:01.523530", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-04-10T07:01:02.760 INFO:teuthology.task.ansible.out:changed: [smithi101.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_1', u'dm-uuid-LVM-WHdv0fpAvhSEIrkkH4emIMiTVjGageHVxRks5xBUIDWaNgUYfbZCI105tDYmDHOw'], u'uuids': [u'7d71e823-548f-40ca-b90c-bfac80e9c924']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-4'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.198748", "end": "2020-04-10 07:01:02.726827", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-WHdv0fpAvhSEIrkkH4emIMiTVjGageHVxRks5xBUIDWaNgUYfbZCI105tDYmDHOw"], "labels": [], "masters": [], "uuids": ["7d71e823-548f-40ca-b90c-bfac80e9c924"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-04-10 07:01:01.528079", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-04-10T07:01:03.794 INFO:teuthology.task.ansible.out:changed: [smithi204.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_3', u'dm-uuid-LVM-qYMUetxG7lg5cwi90Cl7E9dn0vkL495yBpf3DVVT3rxE6Ib2qGz7Wdtc81izvSnR'], u'uuids': [u'2e28b19c-603b-4f10-b06e-f59820b53a3c']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-2'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.138391", "end": "2020-04-10 07:01:03.966330", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-qYMUetxG7lg5cwi90Cl7E9dn0vkL495yBpf3DVVT3rxE6Ib2qGz7Wdtc81izvSnR"], "labels": [], "masters": [], "uuids": ["2e28b19c-603b-4f10-b06e-f59820b53a3c"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-04-10 07:01:02.827939", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-04-10T07:01:04.029 INFO:teuthology.task.ansible.out:changed: [smithi101.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_3', u'dm-uuid-LVM-WHdv0fpAvhSEIrkkH4emIMiTVjGageHVgzVa14chrqOXDVnFrYTpydriEkjFP9GU'], u'uuids': [u'a8e6f815-2bf1-434c-aa77-5b609c74afec']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-2'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.136740", "end": "2020-04-10 07:01:03.997611", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-WHdv0fpAvhSEIrkkH4emIMiTVjGageHVgzVa14chrqOXDVnFrYTpydriEkjFP9GU"], "labels": [], "masters": [], "uuids": ["a8e6f815-2bf1-434c-aa77-5b609c74afec"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-04-10 07:01:02.860871", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-04-10T07:01:05.062 INFO:teuthology.task.ansible.out:changed: [smithi204.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_2', u'dm-uuid-LVM-qYMUetxG7lg5cwi90Cl7E9dn0vkL495yT4750rPNNdRQ64G49LrKffldsxNfvuM8'], u'uuids': [u'e987e8fc-53c5-4322-b1d3-091e2ba818df']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-3'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.137172", "end": "2020-04-10 07:01:05.232813", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-qYMUetxG7lg5cwi90Cl7E9dn0vkL495yT4750rPNNdRQ64G49LrKffldsxNfvuM8"], "labels": [], "masters": [], "uuids": ["e987e8fc-53c5-4322-b1d3-091e2ba818df"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-04-10 07:01:04.095641", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-04-10T07:01:05.353 INFO:teuthology.task.ansible.out:changed: [smithi101.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_2', u'dm-uuid-LVM-WHdv0fpAvhSEIrkkH4emIMiTVjGageHVclbn2NTC29xrfCfee9rYNj6lv084ptao'], u'uuids': [u'8fafe4be-ba09-4873-a061-85a371d44889']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-3'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.190024", "end": "2020-04-10 07:01:05.321595", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-WHdv0fpAvhSEIrkkH4emIMiTVjGageHVclbn2NTC29xrfCfee9rYNj6lv084ptao"], "labels": [], "masters": [], "uuids": ["8fafe4be-ba09-4873-a061-85a371d44889"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-04-10 07:01:04.131571", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-04-10T07:01:06.321 INFO:teuthology.task.ansible.out:changed: [smithi204.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_5', u'dm-uuid-LVM-qYMUetxG7lg5cwi90Cl7E9dn0vkL495yfFmfgdxMVc9EbxXLLX91ntqCx53vbYzn'], u'uuids': [u'7e40015a-cd9e-4e0b-aeda-c3bbd5717614']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'14.90 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'31252480', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-0'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.127377", "end": "2020-04-10 07:01:06.482183", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-qYMUetxG7lg5cwi90Cl7E9dn0vkL495yfFmfgdxMVc9EbxXLLX91ntqCx53vbYzn"], "labels": [], "masters": [], "uuids": ["7e40015a-cd9e-4e0b-aeda-c3bbd5717614"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-04-10 07:01:05.354806", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-04-10T07:01:06.617 INFO:teuthology.task.ansible.out:changed: [smithi101.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_5', u'dm-uuid-LVM-WHdv0fpAvhSEIrkkH4emIMiTVjGageHViPC3Xvkh3coH44urrhi5vxGzYdhNy0GE'], u'uuids': [u'f5f1e14b-d6e4-42a1-a9cd-db2e28fc5300']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'14.90 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'31252480', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-0'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.144121", "end": "2020-04-10 07:01:06.596785", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-WHdv0fpAvhSEIrkkH4emIMiTVjGageHViPC3Xvkh3coH44urrhi5vxGzYdhNy0GE"], "labels": [], "masters": [], "uuids": ["f5f1e14b-d6e4-42a1-a9cd-db2e28fc5300"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-04-10 07:01:05.452664", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-04-10T07:01:07.592 INFO:teuthology.task.ansible.out:changed: [smithi204.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_4', u'dm-uuid-LVM-qYMUetxG7lg5cwi90Cl7E9dn0vkL495yJWyHyUNq7KEU0cmYW202u5Wat0jcDaR0'], u'uuids': [u'82f0d051-df68-4c9a-85f8-5ad3c7d0bfbb']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.133927", "end": "2020-04-10 07:01:07.757057", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-qYMUetxG7lg5cwi90Cl7E9dn0vkL495yJWyHyUNq7KEU0cmYW202u5Wat0jcDaR0"], "labels": [], "masters": [], "uuids": ["82f0d051-df68-4c9a-85f8-5ad3c7d0bfbb"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-04-10 07:01:06.623130", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-04-10T07:01:07.666 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:07 +0000 (0:00:08.214) 0:02:18.119 ********** 2020-04-10T07:01:07.724 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-04-10T07:01:07.725 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': u'ATA', u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W5XP2E', u'wwn-0x5000c50092941a36'], u'uuids': []}, u'sas_device_handle': None, u'host': u'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', u'support_discard': u'0', u'serial': u'Z1W5XP2E', u'wwn': u'0x5000c50092941a36', u'holders': [], u'size': u'931.51 GB', u'scheduler_mode': u'cfq', u'rotational': u'1', u'sectors': u'1953525168', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'ST1000NM0033-9ZM', u'partitions': {u'sda1': {u'start': u'2048', u'sectorsize': 512, u'uuid': u'298ac6f2-5334-4198-9d30-0049606b867a', u'sectors': u'1953522688', u'holders': [], u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W5XP2E-part1', u'wwn-0x5000c50092941a36-part1'], u'uuids': [u'298ac6f2-5334-4198-9d30-0049606b867a']}, u'size': u'931.51 GB'}}}, 'key': u'sda'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XP2E", "wwn-0x5000c50092941a36"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XP2E-part1", "wwn-0x5000c50092941a36-part1"], "labels": [], "masters": [], "uuids": ["298ac6f2-5334-4198-9d30-0049606b867a"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "298ac6f2-5334-4198-9d30-0049606b867a"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5XP2E", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "vir 2020-04-10T07:01:07.727 INFO:teuthology.task.ansible.out:tual": 1, "wwn": "0x5000c50092941a36"}}, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:07.738 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'0', u'model': None, u'size': u'0.00 Bytes', u'scheduler_mode': u'none', u'rotational': u'1', u'sectors': u'0', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'loop3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:07.751 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'0', u'model': None, u'size': u'0.00 Bytes', u'scheduler_mode': u'none', u'rotational': u'1', u'sectors': u'0', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'loop2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:07.762 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'0', u'model': None, u'size': u'0.00 Bytes', u'scheduler_mode': u'none', u'rotational': u'1', u'sectors': u'0', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'loop1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:07.764 INFO:teuthology.task.ansible.out: 2020-04-10T07:01:07.774 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'0', u'model': None, u'size': u'0.00 Bytes', u'scheduler_mode': u'none', u'rotational': u'1', u'sectors': u'0', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'loop0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:07.793 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'0', u'model': None, u'size': u'0.00 Bytes', u'scheduler_mode': u'none', u'rotational': u'1', u'sectors': u'0', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'loop7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:07.800 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'0', u'model': None, u'size': u'0.00 Bytes', u'scheduler_mode': u'none', u'rotational': u'1', u'sectors': u'0', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'loop6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:07.814 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'0', u'model': None, u'size': u'0.00 Bytes', u'scheduler_mode': u'none', u'rotational': u'1', u'sectors': u'0', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'loop5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:07.827 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'0', u'model': None, u'size': u'0.00 Bytes', u'scheduler_mode': u'none', u'rotational': u'1', u'sectors': u'0', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'loop4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:07.915 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-04-10T07:01:07.916 INFO:teuthology.task.ansible.out:changed: [smithi101.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_4', u'dm-uuid-LVM-WHdv0fpAvhSEIrkkH4emIMiTVjGageHVS1xRBoG09v5fon1vzCTI2O5lU9gCuc2M'], u'uuids': [u'5e23b0e6-2e46-4020-a9ad-5c4a904587b0']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.169221", "end": "2020-04-10 07:01:07.889093", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-WHdv0fpAvhSEIrkkH4emIMiTVjGageHVS1xRBoG09v5fon1vzCTI2O5lU9gCuc2M"], "labels": [], "masters": [], "uuids": ["5e23b0e6-2e46-4020-a9ad-5c4a904587b0"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-04-10 07:01:06.719872", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-04-10T07:01:08.001 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:07 +0000 (0:00:00.334) 0:02:18.454 ********** 2020-04-10T07:01:08.032 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** changed: [smithi204.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [u'dm-0', u'dm-1', u'dm-2', u'dm-3', u'dm-4'], u'labels': [], u'ids': [u'lvm-pv-uuid-xQ1879-oYTb-0ddJ-eCMO-Cc4n-UdsM-U29Hj5', u'nvme-INTEL_SSDPEDMD400G4_CVFT623300GD400BGN', u'nvme-nvme.8086-43564654363233333030474434303042474e-494e54454c205353445045444d443430304734-00000001'], u'uuids': []}, u'sas_device_handle': None, u'host': u'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', u'support_discard': u'512', u'model': u'INTEL SSDPEDMD400G4', u'size': u'372.61 GB', u'scheduler_mode': u'none', u'rotational': u'0', u'sectors': u'781422768', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [u'vg_nvme-lv_4', u'vg_nvme-lv_1', u'vg_nvme-lv_3', u'vg_nvme-lv_5', u'vg_nvme-lv_2'], u'partitions': {}}, 'key': u'nvme0n1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.049924", "end": "2020-04-10 07:01:08.170450", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_4", "vg_nvme-lv_1", "vg_nvme-lv_3", "vg_nvme-lv_5", "vg_nvme-lv_2"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-xQ1879-oYTb-0ddJ-eCMO-Cc4n-UdsM-U29Hj5", "nvme-INTEL_SSDPEDMD400G4_CVFT623300GD400BGN", "nvme-nvme.8086-43564654363233333030474434303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-04-10 07:01:08. 2020-04-10T07:01:08.034 INFO:teuthology.task.ansible.out:120526", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0343133 s, 3.4 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0343133 s, 3.4 GB/s"], "stdout": "", "stdout_lines": []} 2020-04-10T07:01:08.091 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': u'ATA', u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W5QCGY', u'wwn-0x5000c50091e05c18'], u'uuids': []}, u'sas_device_handle': None, u'host': u'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', u'support_discard': u'0', u'serial': u'Z1W5QCGY', u'wwn': u'0x5000c50091e05c18', u'holders': [], u'size': u'931.51 GB', u'scheduler_mode': u'cfq', u'rotational': u'1', u'sectors': u'1953525168', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'ST1000NM0033-9ZM', u'partitions': {u'sda1': {u'start': u'2048', u'sectorsize': 512, u'uuid': u'298ac6f2-5334-4198-9d30-0049606b867a', u'sectors': u'1953522688', u'holders': [], u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W5QCGY-part1', u'wwn-0x5000c50091e05c18-part1'], u'uuids': [u'298ac6f2-5334-4198-9d30-0049606b867a']}, u'size': u'931.51 GB'}}}, 'key': u'sda'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QCGY", "wwn-0x5000c50091e05c18"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QCGY-part1", "wwn-0x5000c50091e05c18-part1"], "labels": [], "masters": [], "uuids": ["298ac6f2-5334-4198-9d30-0049606b867a"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "298ac6f2-5334-4198-9d30-0049606b867a"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5QCGY", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "vir 2020-04-10T07:01:08.093 INFO:teuthology.task.ansible.out:tual": 1, "wwn": "0x5000c50091e05c18"}}, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:08.095 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'0', u'model': None, u'size': u'0.00 Bytes', u'scheduler_mode': u'none', u'rotational': u'1', u'sectors': u'0', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'loop3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:08.106 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'0', u'model': None, u'size': u'0.00 Bytes', u'scheduler_mode': u'none', u'rotational': u'1', u'sectors': u'0', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'loop2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:08.130 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'0', u'model': None, u'size': u'0.00 Bytes', u'scheduler_mode': u'none', u'rotational': u'1', u'sectors': u'0', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'loop1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:08.146 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'0', u'model': None, u'size': u'0.00 Bytes', u'scheduler_mode': u'none', u'rotational': u'1', u'sectors': u'0', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'loop0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:08.160 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'0', u'model': None, u'size': u'0.00 Bytes', u'scheduler_mode': u'none', u'rotational': u'1', u'sectors': u'0', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'loop7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:08.181 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'0', u'model': None, u'size': u'0.00 Bytes', u'scheduler_mode': u'none', u'rotational': u'1', u'sectors': u'0', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'loop6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:08.190 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'0', u'model': None, u'size': u'0.00 Bytes', u'scheduler_mode': u'none', u'rotational': u'1', u'sectors': u'0', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'loop5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:08.193 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'0', u'model': None, u'size': u'0.00 Bytes', u'scheduler_mode': u'none', u'rotational': u'1', u'sectors': u'0', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'loop4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:08.345 INFO:teuthology.task.ansible.out:changed: [smithi204.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_1', u'dm-uuid-LVM-qYMUetxG7lg5cwi90Cl7E9dn0vkL495ycD1SQr1DrycguuuXSTFBH6fnExcFcRcA'], u'uuids': [u'9601b024-cb25-4d3f-9528-e5def0570c6f']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-4'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.157948", "end": "2020-04-10 07:01:08.513051", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-qYMUetxG7lg5cwi90Cl7E9dn0vkL495ycD1SQr1DrycguuuXSTFBH6fnExcFcRcA"], "labels": [], "masters": [], "uuids": ["9601b024-cb25-4d3f-9528-e5def0570c6f"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-04-10 07:01:08.355103", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.155461 s, 742 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.155461 s, 742 MB/s"], "stdout": "", "stdout_lines": []} 2020-04-10T07:01:08.400 INFO:teuthology.task.ansible.out:changed: [smithi101.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [u'dm-0', u'dm-1', u'dm-2', u'dm-3', u'dm-4'], u'labels': [], u'ids': [u'lvm-pv-uuid-AWWcD4-GqJ9-WTUT-MhbF-0NfY-1fdC-21akW9', u'nvme-INTEL_SSDPEDMD400G4_PHFT6204017B400BGN', u'nvme-nvme.8086-50484654363230343031374234303042474e-494e54454c205353445045444d443430304734-00000001'], u'uuids': []}, u'sas_device_handle': None, u'host': u'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', u'support_discard': u'512', u'model': u'INTEL SSDPEDMD400G4', u'size': u'372.61 GB', u'scheduler_mode': u'none', u'rotational': u'0', u'sectors': u'781422768', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [u'vg_nvme-lv_4', u'vg_nvme-lv_1', u'vg_nvme-lv_3', u'vg_nvme-lv_5', u'vg_nvme-lv_2'], u'partitions': {}}, 'key': u'nvme0n1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.058707", "end": "2020-04-10 07:01:08.366348", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_4", "vg_nvme-lv_1", "vg_nvme-lv_3", "vg_nvme-lv_5", "vg_nvme-lv_2"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-AWWcD4-GqJ9-WTUT-MhbF-0NfY-1fdC-21akW9", "nvme-INTEL_SSDPEDMD400G4_PHFT6204017B400BGN", "nvme-nvme.8086-50484654363230343031374234303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-04-10 07:01:08.307641", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 11 2020-04-10T07:01:08.402 INFO:teuthology.task.ansible.out:0 MiB) copied, 0.0408281 s, 2.8 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0408281 s, 2.8 GB/s"], "stdout": "", "stdout_lines": []} 2020-04-10T07:01:08.632 INFO:teuthology.task.ansible.out:changed: [smithi204.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_3', u'dm-uuid-LVM-qYMUetxG7lg5cwi90Cl7E9dn0vkL495yBpf3DVVT3rxE6Ib2qGz7Wdtc81izvSnR'], u'uuids': [u'2e28b19c-603b-4f10-b06e-f59820b53a3c']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-2'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.144222", "end": "2020-04-10 07:01:08.797874", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-qYMUetxG7lg5cwi90Cl7E9dn0vkL495yBpf3DVVT3rxE6Ib2qGz7Wdtc81izvSnR"], "labels": [], "masters": [], "uuids": ["2e28b19c-603b-4f10-b06e-f59820b53a3c"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-04-10 07:01:08.653652", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.140455 s, 821 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.140455 s, 821 MB/s"], "stdout": "", "stdout_lines": []} 2020-04-10T07:01:08.667 INFO:teuthology.task.ansible.out:changed: [smithi101.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_1', u'dm-uuid-LVM-WHdv0fpAvhSEIrkkH4emIMiTVjGageHVxRks5xBUIDWaNgUYfbZCI105tDYmDHOw'], u'uuids': [u'7d71e823-548f-40ca-b90c-bfac80e9c924']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-4'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.143160", "end": "2020-04-10 07:01:08.645303", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-WHdv0fpAvhSEIrkkH4emIMiTVjGageHVxRks5xBUIDWaNgUYfbZCI105tDYmDHOw"], "labels": [], "masters": [], "uuids": ["7d71e823-548f-40ca-b90c-bfac80e9c924"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-04-10 07:01:08.502143", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.140351 s, 822 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.140351 s, 822 MB/s"], "stdout": "", "stdout_lines": []} 2020-04-10T07:01:08.923 INFO:teuthology.task.ansible.out:changed: [smithi204.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_2', u'dm-uuid-LVM-qYMUetxG7lg5cwi90Cl7E9dn0vkL495yT4750rPNNdRQ64G49LrKffldsxNfvuM8'], u'uuids': [u'e987e8fc-53c5-4322-b1d3-091e2ba818df']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-3'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.162929", "end": "2020-04-10 07:01:09.085222", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-qYMUetxG7lg5cwi90Cl7E9dn0vkL495yT4750rPNNdRQ64G49LrKffldsxNfvuM8"], "labels": [], "masters": [], "uuids": ["e987e8fc-53c5-4322-b1d3-091e2ba818df"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-04-10 07:01:08.922293", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.15825 s, 729 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.15825 s, 729 MB/s"], "stdout": "", "stdout_lines": []} 2020-04-10T07:01:08.946 INFO:teuthology.task.ansible.out:changed: [smithi101.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_3', u'dm-uuid-LVM-WHdv0fpAvhSEIrkkH4emIMiTVjGageHVgzVa14chrqOXDVnFrYTpydriEkjFP9GU'], u'uuids': [u'a8e6f815-2bf1-434c-aa77-5b609c74afec']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-2'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.153836", "end": "2020-04-10 07:01:08.922442", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-WHdv0fpAvhSEIrkkH4emIMiTVjGageHVgzVa14chrqOXDVnFrYTpydriEkjFP9GU"], "labels": [], "masters": [], "uuids": ["a8e6f815-2bf1-434c-aa77-5b609c74afec"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-04-10 07:01:08.768606", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.149241 s, 773 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.149241 s, 773 MB/s"], "stdout": "", "stdout_lines": []} 2020-04-10T07:01:09.187 INFO:teuthology.task.ansible.out:changed: [smithi204.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_5', u'dm-uuid-LVM-qYMUetxG7lg5cwi90Cl7E9dn0vkL495yfFmfgdxMVc9EbxXLLX91ntqCx53vbYzn'], u'uuids': [u'7e40015a-cd9e-4e0b-aeda-c3bbd5717614']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'14.90 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'31252480', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-0'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.145594", "end": "2020-04-10 07:01:09.350012", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-qYMUetxG7lg5cwi90Cl7E9dn0vkL495yfFmfgdxMVc9EbxXLLX91ntqCx53vbYzn"], "labels": [], "masters": [], "uuids": ["7e40015a-cd9e-4e0b-aeda-c3bbd5717614"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-04-10 07:01:09.204418", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.142931 s, 807 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.142931 s, 807 MB/s"], "stdout": "", "stdout_lines": []} 2020-04-10T07:01:09.241 INFO:teuthology.task.ansible.out:changed: [smithi101.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_2', u'dm-uuid-LVM-WHdv0fpAvhSEIrkkH4emIMiTVjGageHVclbn2NTC29xrfCfee9rYNj6lv084ptao'], u'uuids': [u'8fafe4be-ba09-4873-a061-85a371d44889']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-3'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.155808", "end": "2020-04-10 07:01:09.208519", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-WHdv0fpAvhSEIrkkH4emIMiTVjGageHVclbn2NTC29xrfCfee9rYNj6lv084ptao"], "labels": [], "masters": [], "uuids": ["8fafe4be-ba09-4873-a061-85a371d44889"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-04-10 07:01:09.052711", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.151122 s, 763 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.151122 s, 763 MB/s"], "stdout": "", "stdout_lines": []} 2020-04-10T07:01:09.458 INFO:teuthology.task.ansible.out:changed: [smithi204.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_4', u'dm-uuid-LVM-qYMUetxG7lg5cwi90Cl7E9dn0vkL495yJWyHyUNq7KEU0cmYW202u5Wat0jcDaR0'], u'uuids': [u'82f0d051-df68-4c9a-85f8-5ad3c7d0bfbb']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.145550", "end": "2020-04-10 07:01:09.621278", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-qYMUetxG7lg5cwi90Cl7E9dn0vkL495yJWyHyUNq7KEU0cmYW202u5Wat0jcDaR0"], "labels": [], "masters": [], "uuids": ["82f0d051-df68-4c9a-85f8-5ad3c7d0bfbb"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-04-10 07:01:09.475728", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.14174 s, 814 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.14174 s, 814 MB/s"], "stdout": "", "stdout_lines": []} 2020-04-10T07:01:09.521 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:09 +0000 (0:00:01.520) 0:02:19.975 ********** 2020-04-10T07:01:09.562 INFO:teuthology.task.ansible.out:changed: [smithi101.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_5', u'dm-uuid-LVM-WHdv0fpAvhSEIrkkH4emIMiTVjGageHViPC3Xvkh3coH44urrhi5vxGzYdhNy0GE'], u'uuids': [u'f5f1e14b-d6e4-42a1-a9cd-db2e28fc5300']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'14.90 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'31252480', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-0'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.168230", "end": "2020-04-10 07:01:09.530356", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-WHdv0fpAvhSEIrkkH4emIMiTVjGageHViPC3Xvkh3coH44urrhi5vxGzYdhNy0GE"], "labels": [], "masters": [], "uuids": ["f5f1e14b-d6e4-42a1-a9cd-db2e28fc5300"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-04-10 07:01:09.362126", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.163444 s, 706 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.163444 s, 706 MB/s"], "stdout": "", "stdout_lines": []} 2020-04-10T07:01:09.816 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** 2020-04-10T07:01:09.818 INFO:teuthology.task.ansible.out:changed: [smithi204.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.150106", "end": "2020-04-10 07:01:09.981011", "rc": 0, "start": "2020-04-10 07:01:09.830905", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-04-10T07:01:09.900 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:09 +0000 (0:00:00.378) 0:02:20.353 ********** 2020-04-10T07:01:09.906 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-04-10T07:01:09.908 INFO:teuthology.task.ansible.out:changed: [smithi101.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_4', u'dm-uuid-LVM-WHdv0fpAvhSEIrkkH4emIMiTVjGageHVS1xRBoG09v5fon1vzCTI2O5lU9gCuc2M'], u'uuids': [u'5e23b0e6-2e46-4020-a9ad-5c4a904587b0']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.165002", "end": "2020-04-10 07:01:09.826611", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-WHdv0fpAvhSEIrkkH4emIMiTVjGageHVS1xRBoG09v5fon1vzCTI2O5lU9gCuc2M"], "labels": [], "masters": [], "uuids": ["5e23b0e6-2e46-4020-a9ad-5c4a904587b0"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-04-10 07:01:09.661609", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.160228 s, 720 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.160228 s, 720 MB/s"], "stdout": "", "stdout_lines": []} 2020-04-10T07:01:09.969 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:09 +0000 (0:00:00.068) 0:02:20.422 ********** 2020-04-10T07:01:10.092 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2020-04-10T07:01:10.094 INFO:teuthology.task.ansible.out:changed: [smithi204.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.053327", "end": "2020-04-10 07:01:10.261985", "rc": 0, "start": "2020-04-10 07:01:10.208658", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-04-10T07:01:10.133 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:10 +0000 (0:00:00.164) 0:02:20.586 ********** 2020-04-10T07:01:10.159 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2020-04-10T07:01:10.243 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:10 +0000 (0:00:00.110) 0:02:20.696 ********** 2020-04-10T07:01:10.260 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** 2020-04-10T07:01:10.261 INFO:teuthology.task.ansible.out:changed: [smithi101.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.144889", "end": "2020-04-10 07:01:10.232575", "rc": 0, "start": "2020-04-10 07:01:10.087686", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-04-10T07:01:10.308 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:10 +0000 (0:00:00.065) 0:02:20.761 ********** 2020-04-10T07:01:10.315 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2020-04-10T07:01:10.317 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:10.373 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:10 +0000 (0:00:00.064) 0:02:20.826 ********** 2020-04-10T07:01:10.423 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:10.507 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:10 +0000 (0:00:00.130) 0:02:20.956 ********** 2020-04-10T07:01:10.515 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2020-04-10T07:01:10.516 INFO:teuthology.task.ansible.out:changed: [smithi101.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.047286", "end": "2020-04-10 07:01:10.476991", "rc": 0, "start": "2020-04-10 07:01:10.429705", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-04-10T07:01:10.566 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:10 +0000 (0:00:00.062) 0:02:21.019 ********** 2020-04-10T07:01:10.572 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2020-04-10T07:01:10.572 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:10.618 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:10 +0000 (0:00:00.052) 0:02:21.071 ********** 2020-04-10T07:01:10.624 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2020-04-10T07:01:10.695 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:10 +0000 (0:00:00.077) 0:02:21.148 ********** 2020-04-10T07:01:10.704 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2020-04-10T07:01:10.706 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:10.770 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:10 +0000 (0:00:00.074) 0:02:21.223 ********** 2020-04-10T07:01:10.778 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:10.827 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:10 +0000 (0:00:00.056) 0:02:21.280 ********** 2020-04-10T07:01:10.857 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:10.922 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:10 +0000 (0:00:00.094) 0:02:21.374 ********** 2020-04-10T07:01:10.930 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ ok: [smithi204.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-04-10T07:01:10.995 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:10 +0000 (0:00:00.073) 0:02:21.448 ********** 2020-04-10T07:01:11.003 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:11.083 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:11 +0000 (0:00:00.087) 0:02:21.536 ********** 2020-04-10T07:01:11.093 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-04-10T07:01:11.095 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={u'block_used': 13740116, u'uuid': u'298ac6f2-5334-4198-9d30-0049606b867a', u'size_total': 983429562368, u'block_total': 240095108, u'mount': u'/', u'block_available': 226354992, u'size_available': 927150047232, u'fstype': u'ext4', u'inode_total': 61054976, u'inode_available': 60905745, u'device': u'/dev/sda1', u'inode_used': 149231, u'block_size': 4096, u'options': u'rw,relatime,errors=remount-ro,data=ordered'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226354992, "block_size": 4096, "block_total": 240095108, "block_used": 13740116, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905745, "inode_total": 61054976, "inode_used": 149231, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927150047232, "size_total": 983429562368, "uuid": "298ac6f2-5334-4198-9d30-0049606b867a"}, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:11.146 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:11 +0000 (0:00:00.063) 0:02:21.599 ********** 2020-04-10T07:01:11.152 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2020-04-10T07:01:11.153 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:11.196 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:11 +0000 (0:00:00.050) 0:02:21.650 ********** 2020-04-10T07:01:11.203 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2020-04-10T07:01:11.203 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:11.271 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:11 +0000 (0:00:00.072) 0:02:21.722 ********** 2020-04-10T07:01:11.312 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:11.373 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:11 +0000 (0:00:00.100) 0:02:21.822 ********** 2020-04-10T07:01:11.376 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2020-04-10T07:01:11.377 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-04-10T07:01:11.431 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:11 +0000 (0:00:00.062) 0:02:21.884 ********** 2020-04-10T07:01:11.446 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:11.512 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:11 +0000 (0:00:00.079) 0:02:21.964 ********** 2020-04-10T07:01:11.523 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-04-10T07:01:11.524 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={u'block_used': 13740115, u'uuid': u'298ac6f2-5334-4198-9d30-0049606b867a', u'size_total': 983429562368, u'block_total': 240095108, u'mount': u'/', u'block_available': 226354993, u'size_available': 927150051328, u'fstype': u'ext4', u'inode_total': 61054976, u'inode_available': 60905745, u'device': u'/dev/sda1', u'inode_used': 149231, u'block_size': 4096, u'options': u'rw,relatime,errors=remount-ro,data=ordered'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226354993, "block_size": 4096, "block_total": 240095108, "block_used": 13740115, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905745, "inode_total": 61054976, "inode_used": 149231, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927150051328, "size_total": 983429562368, "uuid": "298ac6f2-5334-4198-9d30-0049606b867a"}, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:11.596 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:11 +0000 (0:00:00.085) 0:02:22.049 ********** 2020-04-10T07:01:11.602 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2020-04-10T07:01:11.604 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:11.649 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:11 +0000 (0:00:00.053) 0:02:22.103 ********** 2020-04-10T07:01:11.655 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2020-04-10T07:01:11.657 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:11.732 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:11 +0000 (0:00:00.078) 0:02:22.181 ********** 2020-04-10T07:01:11.734 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2020-04-10T07:01:11.736 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:11.793 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:11 +0000 (0:00:00.065) 0:02:22.246 ********** 2020-04-10T07:01:11.803 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-04-10T07:01:11.805 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:11.857 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:11 +0000 (0:00:00.064) 0:02:22.310 ********** 2020-04-10T07:01:11.880 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:11.922 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:11 +0000 (0:00:00.064) 0:02:22.375 ********** 2020-04-10T07:01:11.930 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-04-10T07:01:11.932 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:11.984 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:11 +0000 (0:00:00.061) 0:02:22.437 ********** 2020-04-10T07:01:12.029 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:12.077 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:12 +0000 (0:00:00.093) 0:02:22.530 ********** 2020-04-10T07:01:12.127 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2020-04-10T07:01:12.129 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:12.209 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:12 +0000 (0:00:00.127) 0:02:22.657 ********** 2020-04-10T07:01:12.243 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2020-04-10T07:01:12.245 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:12.306 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:12 +0000 (0:00:00.101) 0:02:22.759 ********** 2020-04-10T07:01:12.535 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** 2020-04-10T07:01:12.537 INFO:teuthology.task.ansible.out:changed: [smithi204.front.sepia.ceph.com] => (item={'value': {u'pvs': u'/dev/nvme0n1'}, 'key': u'vg_nvme'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2020-04-10T07:01:12.641 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:12 +0000 (0:00:00.333) 0:02:23.092 ********** 2020-04-10T07:01:12.669 INFO:teuthology.task.ansible.out:changed: [smithi101.front.sepia.ceph.com] => (item={'value': {u'pvs': u'/dev/nvme0n1'}, 'key': u'vg_nvme'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2020-04-10T07:01:12.729 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:12 +0000 (0:00:00.089) 0:02:23.182 ********** 2020-04-10T07:01:13.325 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-04-10T07:01:13.327 INFO:teuthology.task.ansible.out:changed: [smithi204.front.sepia.ceph.com] => (item={'value': {u'vg': u'vg_nvme', u'size': u'14.9G'}, 'key': u'lv_5'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2020-04-10T07:01:13.348 INFO:teuthology.task.ansible.out:changed: [smithi101.front.sepia.ceph.com] => (item={'value': {u'vg': u'vg_nvme', u'size': u'14.9G'}, 'key': u'lv_5'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2020-04-10T07:01:13.555 INFO:teuthology.task.ansible.out:changed: [smithi204.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_4'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-04-10T07:01:13.611 INFO:teuthology.task.ansible.out:changed: [smithi101.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_4'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-04-10T07:01:13.806 INFO:teuthology.task.ansible.out:changed: [smithi204.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_3'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-04-10T07:01:13.906 INFO:teuthology.task.ansible.out:changed: [smithi101.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_3'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-04-10T07:01:14.092 INFO:teuthology.task.ansible.out:changed: [smithi204.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_2'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-04-10T07:01:14.182 INFO:teuthology.task.ansible.out:changed: [smithi101.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_2'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-04-10T07:01:14.387 INFO:teuthology.task.ansible.out:changed: [smithi204.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_1'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-04-10T07:01:14.440 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:14 +0000 (0:00:01.711) 0:02:24.893 ********** 2020-04-10T07:01:14.457 INFO:teuthology.task.ansible.out:changed: [smithi101.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_1'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-04-10T07:01:14.501 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:14 +0000 (0:00:00.060) 0:02:24.954 ********** 2020-04-10T07:01:14.604 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** changed: [smithi204.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2020-04-10T07:01:14.669 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:14 +0000 (0:00:00.168) 0:02:25.122 ********** 2020-04-10T07:01:14.678 INFO:teuthology.task.ansible.out:changed: [smithi101.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2020-04-10T07:01:14.763 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:14 +0000 (0:00:00.089) 0:02:25.212 ********** 2020-04-10T07:01:14.768 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2020-04-10T07:01:14.768 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={'value': {u'vg': u'vg_nvme', u'size': u'14.9G'}, 'key': u'lv_5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:14.836 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={'value': {u'vg': u'vg_nvme', u'size': u'14.9G'}, 'key': u'lv_5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:14.901 INFO:teuthology.task.ansible.out:changed: [smithi204.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_4'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-04-10T07:01:14.981 INFO:teuthology.task.ansible.out:changed: [smithi101.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_4'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-04-10T07:01:15.031 INFO:teuthology.task.ansible.out:changed: [smithi204.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_3'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-04-10T07:01:15.130 INFO:teuthology.task.ansible.out:changed: [smithi101.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_3'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-04-10T07:01:15.233 INFO:teuthology.task.ansible.out:changed: [smithi204.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_2'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-04-10T07:01:15.292 INFO:teuthology.task.ansible.out:changed: [smithi101.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_2'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-04-10T07:01:15.380 INFO:teuthology.task.ansible.out:changed: [smithi204.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_1'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-04-10T07:01:15.457 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:15 +0000 (0:00:00.696) 0:02:25.909 ********** 2020-04-10T07:01:15.467 INFO:teuthology.task.ansible.out:changed: [smithi101.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_1'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-04-10T07:01:15.518 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:15 +0000 (0:00:00.060) 0:02:25.970 ********** 2020-04-10T07:01:15.609 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** 2020-04-10T07:01:15.611 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "size": 4096, "state": "directory", "uid": 0} 2020-04-10T07:01:15.706 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:15 +0000 (0:00:00.186) 0:02:26.157 ********** 2020-04-10T07:01:15.726 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "size": 4096, "state": "directory", "uid": 0} 2020-04-10T07:01:15.809 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:15 +0000 (0:00:00.098) 0:02:26.255 ********** 2020-04-10T07:01:15.813 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** ok: [smithi204.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2020-04-10T07:01:15.893 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:15 +0000 (0:00:00.090) 0:02:26.346 ********** 2020-04-10T07:01:15.913 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2020-04-10T07:01:15.969 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:15 +0000 (0:00:00.075) 0:02:26.422 ********** 2020-04-10T07:01:16.434 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* changed: [smithi101.front.sepia.ceph.com] => {"changed": true} 2020-04-10T07:01:16.497 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:16 +0000 (0:00:00.526) 0:02:26.949 ********** 2020-04-10T07:01:16.503 INFO:teuthology.task.ansible.out:changed: [smithi204.front.sepia.ceph.com] => {"changed": true} 2020-04-10T07:01:16.545 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:16 +0000 (0:00:00.049) 0:02:26.998 ********** 2020-04-10T07:01:16.989 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* 2020-04-10T07:01:16.990 INFO:teuthology.task.ansible.out:changed: [smithi204.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2020-04-10T07:01:17.048 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:17 +0000 (0:00:00.502) 0:02:27.501 ********** 2020-04-10T07:01:17.089 INFO:teuthology.task.ansible.out:changed: [smithi101.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2020-04-10T07:01:17.160 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:17 +0000 (0:00:00.111) 0:02:27.613 ********** 2020-04-10T07:01:17.225 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** 2020-04-10T07:01:17.228 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => {"changed": false, "gid": 1211, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 1000} 2020-04-10T07:01:17.280 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:17 +0000 (0:00:00.120) 0:02:27.733 ********** 2020-04-10T07:01:17.338 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => {"changed": false, "gid": 1211, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 1000} 2020-04-10T07:01:17.394 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:17 +0000 (0:00:00.114) 0:02:27.847 ********** 2020-04-10T07:01:17.433 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2020-04-10T07:01:17.434 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 0} 2020-04-10T07:01:17.494 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:17 +0000 (0:00:00.099) 0:02:27.947 ********** 2020-04-10T07:01:17.561 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 0} 2020-04-10T07:01:17.671 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:17 +0000 (0:00:00.176) 0:02:28.124 ********** 2020-04-10T07:01:17.872 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** 2020-04-10T07:01:17.873 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "gid": 1211, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 1000} 2020-04-10T07:01:17.932 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:17 +0000 (0:00:00.258) 0:02:28.383 ********** 2020-04-10T07:01:18.046 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "gid": 1211, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 1000} 2020-04-10T07:01:18.116 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:18 +0000 (0:00:00.184) 0:02:28.568 ********** 2020-04-10T07:01:18.289 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* 2020-04-10T07:01:18.290 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/root/.cpan/CPAN/MyConfig.pm", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 0} 2020-04-10T07:01:18.356 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:18 +0000 (0:00:00.240) 0:02:28.809 ********** 2020-04-10T07:01:18.469 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/root/.cpan/CPAN/MyConfig.pm", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 0} 2020-04-10T07:01:18.516 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:18 +0000 (0:00:00.159) 0:02:28.969 ********** 2020-04-10T07:01:19.276 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** ok: [smithi204.front.sepia.ceph.com] => {"cache_update_time": 1586501758, "cache_updated": false, "changed": false} 2020-04-10T07:01:19.317 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:19 +0000 (0:00:00.801) 0:02:29.770 ********** 2020-04-10T07:01:19.421 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => {"cache_update_time": 1586501716, "cache_updated": false, "changed": false} 2020-04-10T07:01:19.468 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:19 +0000 (0:00:00.151) 0:02:29.921 ********** 2020-04-10T07:01:19.508 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2020-04-10T07:01:19.509 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => {"changed": false, "cmd": ["perldoc", "-l", "Amazon::S3"], "delta": "0:00:00.043396", "end": "2020-04-10 07:01:19.668230", "rc": 0, "start": "2020-04-10 07:01:19.624834", "stderr": "", "stderr_lines": [], "stdout": "/usr/local/share/perl/5.26.1/Amazon/S3.pm", "stdout_lines": ["/usr/local/share/perl/5.26.1/Amazon/S3.pm"]} 2020-04-10T07:01:19.578 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:19 +0000 (0:00:00.108) 0:02:30.030 ********** 2020-04-10T07:01:19.617 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** 2020-04-10T07:01:19.619 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:19.682 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:19 +0000 (0:00:00.104) 0:02:30.135 ********** 2020-04-10T07:01:19.687 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2020-04-10T07:01:19.689 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => {"changed": false, "cmd": ["perldoc", "-l", "Amazon::S3"], "delta": "0:00:00.046459", "end": "2020-04-10 07:01:19.632254", "rc": 0, "start": "2020-04-10 07:01:19.585795", "stderr": "", "stderr_lines": [], "stdout": "/usr/local/share/perl/5.26.1/Amazon/S3.pm", "stdout_lines": ["/usr/local/share/perl/5.26.1/Amazon/S3.pm"]} 2020-04-10T07:01:19.734 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:19 +0000 (0:00:00.052) 0:02:30.187 ********** 2020-04-10T07:01:19.740 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** 2020-04-10T07:01:19.742 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:19.783 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:19 +0000 (0:00:00.049) 0:02:30.236 ********** 2020-04-10T07:01:19.789 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** 2020-04-10T07:01:19.789 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:19.835 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:19 +0000 (0:00:00.052) 0:02:30.288 ********** 2020-04-10T07:01:19.862 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** 2020-04-10T07:01:19.863 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:19.902 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:19 +0000 (0:00:00.066) 0:02:30.355 ********** 2020-04-10T07:01:20.626 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2020-04-10T07:01:20.627 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => {"cache_update_time": 1586501758, "cache_updated": false, "changed": false} 2020-04-10T07:01:20.677 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:20 +0000 (0:00:00.771) 0:02:31.126 ********** 2020-04-10T07:01:20.702 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:20.759 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:20 +0000 (0:00:00.085) 0:02:31.212 ********** 2020-04-10T07:01:20.765 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2020-04-10T07:01:20.769 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => {"cache_update_time": 1586501716, "cache_updated": false, "changed": false} 2020-04-10T07:01:20.807 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:20 +0000 (0:00:00.047) 0:02:31.260 ********** 2020-04-10T07:01:20.840 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:20.887 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:20 +0000 (0:00:00.080) 0:02:31.340 ********** 2020-04-10T07:01:21.130 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ ok: [smithi204.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "size": 2767, "state": "file", "uid": 0} 2020-04-10T07:01:21.185 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:21 +0000 (0:00:00.296) 0:02:31.637 ********** 2020-04-10T07:01:21.212 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:21.255 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:21 +0000 (0:00:00.071) 0:02:31.708 ********** 2020-04-10T07:01:21.285 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ ok: [smithi101.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "size": 2767, "state": "file", "uid": 0} 2020-04-10T07:01:21.317 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:21 +0000 (0:00:00.061) 0:02:31.770 ********** 2020-04-10T07:01:21.346 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:21.389 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:21 +0000 (0:00:00.070) 0:02:31.841 ********** 2020-04-10T07:01:21.502 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2020-04-10T07:01:21.504 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "ntp", "state": "started", "status": {"ActiveEnterTimestamp": "Fri 2020-04-10 06:58:46 UTC", "ActiveEnterTimestampMonotonic": "26125906", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "sysinit.target -.mount system.slice basic.target systemd-tmpfiles-setup.service systemd-journald.socket network.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Fri 2020-04-10 06:58:38 UTC", "AssertTimestampMonotonic": "17760043", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Fri 2020-04-10 06:58:38 UTC", "ConditionTimestampMonotonic": "17760043", "ConfigurationDirectoryMode": "0755", "Conflicts": "systemd-timesyncd.service shutdown.target", "ControlGroup": "/system.slice/ntp.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "Documen 2020-04-10T07:01:21.506 INFO:teuthology.task.ansible.out:tation": "man:ntpd(8)", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1851", "ExecMainStartTimestamp": "Fri 2020-04-10 06:58:46 UTC", "ExecMainStartTimestampMonotonic": "26125860", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/lib/ntp/ntp-systemd-wrapper ; argv[]=/usr/lib/ntp/ntp-systemd-wrapper ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/ntp.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "ntp.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Fri 2020-04-10 06:58:38 UTC", "InactiveExitTimestampMonotonic": "17760761", "InvocationID": "5b12e5c0c59b4870bdced8f2180b3bbe", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128031", "LimitNPROCSoft": "128031", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "Limi 2020-04-10T07:01:21.507 INFO:teuthology.task.ansible.out:tRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128031", "LimitSIGPENDINGSoft": "128031", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1851", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "ntp.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target -.mount system.slice", "RequiresMountsFor": "/var/tmp /tmp", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Fri 2020-04-10 06:58:46 UTC", "S 2020-04-10T07:01:21.509 INFO:teuthology.task.ansible.out:tateChangeTimestampMonotonic": "26125906", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "2", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Fri 2020-04-10 06:58:46 UTC", "WatchdogTimestampMonotonic": "26125903", "WatchdogUSec": "0"}} 2020-04-10T07:01:21.554 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:21 +0000 (0:00:00.166) 0:02:32.007 ********** 2020-04-10T07:01:21.592 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* 2020-04-10T07:01:21.595 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:21.641 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:21 +0000 (0:00:00.080) 0:02:32.088 ********** 2020-04-10T07:01:21.644 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** ok: [smithi101.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "ntp", "state": "started", "status": {"ActiveEnterTimestamp": "Fri 2020-04-10 06:58:46 UTC", "ActiveEnterTimestampMonotonic": "25483953", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "sysinit.target network.target systemd-tmpfiles-setup.service system.slice -.mount systemd-journald.socket basic.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Fri 2020-04-10 06:58:37 UTC", "AssertTimestampMonotonic": "17112076", "Before": "multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Fri 2020-04-10 06:58:37 UTC", "ConditionTimestampMonotonic": "17112075", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target systemd-timesyncd.service", "ControlGroup": "/system.slice/ntp.service", "ControlPID": "0", "DefaultDependencies": "yes", "Dele 2020-04-10T07:01:21.646 INFO:teuthology.task.ansible.out:gate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "Documentation": "man:ntpd(8)", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "2004", "ExecMainStartTimestamp": "Fri 2020-04-10 06:58:46 UTC", "ExecMainStartTimestampMonotonic": "25483940", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/lib/ntp/ntp-systemd-wrapper ; argv[]=/usr/lib/ntp/ntp-systemd-wrapper ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/ntp.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "ntp.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Fri 2020-04-10 06:58:37 UTC", "InactiveExitTimestampMonotonic": "17112938", "InvocationID": "3228e2f4db234015befb6ae4d06ec8ac", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128031", "LimitNPROCSoft": "128031", "LimitRSS": "infi 2020-04-10T07:01:21.647 INFO:teuthology.task.ansible.out:nity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128031", "LimitSIGPENDINGSoft": "128031", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "2004", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "ntp.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice -.mount sysinit.target", "RequiresMountsFor": "/var/tmp /tmp", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "Star 2020-04-10T07:01:21.653 INFO:teuthology.task.ansible.out:tupIOWeight": "[not set]", "StateChangeTimestamp": "Fri 2020-04-10 06:58:46 UTC", "StateChangeTimestampMonotonic": "25483953", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "2", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Fri 2020-04-10 06:58:46 UTC", "WatchdogTimestampMonotonic": "25483951", "WatchdogUSec": "0"}} 2020-04-10T07:01:21.691 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:21 +0000 (0:00:00.054) 0:02:32.143 ********** 2020-04-10T07:01:21.717 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:21.757 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:21 +0000 (0:00:00.067) 0:02:32.210 ********** 2020-04-10T07:01:22.531 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* ok: [smithi204.front.sepia.ceph.com] => {"cache_update_time": 1586501758, "cache_updated": false, "changed": false} 2020-04-10T07:01:22.577 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:22 +0000 (0:00:00.818) 0:02:33.028 ********** 2020-04-10T07:01:22.603 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:22.649 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:22 +0000 (0:00:00.071) 0:02:33.100 ********** 2020-04-10T07:01:22.663 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* 2020-04-10T07:01:22.664 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => {"cache_update_time": 1586501716, "cache_updated": false, "changed": false} 2020-04-10T07:01:22.710 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:22 +0000 (0:00:00.063) 0:02:33.164 ********** 2020-04-10T07:01:22.738 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:22.786 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:22 +0000 (0:00:00.075) 0:02:33.239 ********** 2020-04-10T07:01:22.793 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** 2020-04-10T07:01:22.794 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => {"changed": false, "gid": 1211, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "size": 4096, "state": "directory", "uid": 1000} 2020-04-10T07:01:22.841 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:22 +0000 (0:00:00.054) 0:02:33.293 ********** 2020-04-10T07:01:22.924 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => {"changed": false, "gid": 1211, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "size": 4096, "state": "directory", "uid": 1000} 2020-04-10T07:01:22.969 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:22 +0000 (0:00:00.128) 0:02:33.422 ********** 2020-04-10T07:01:23.184 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2020-04-10T07:01:23.185 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1211, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "size": 73, "state": "file", "uid": 1000} 2020-04-10T07:01:23.235 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:23 +0000 (0:00:00.266) 0:02:33.688 ********** 2020-04-10T07:01:23.315 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1211, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "size": 73, "state": "file", "uid": 1000} 2020-04-10T07:01:23.364 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:23 +0000 (0:00:00.127) 0:02:33.816 ********** 2020-04-10T07:01:24.347 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* 2020-04-10T07:01:24.348 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => {"changed": false, "cmd": ["/usr/bin/pip2", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "", "stderr_lines": [], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages\nRequirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages", "Requirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2020-04-10T07:01:24.395 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:24 +0000 (0:00:01.031) 0:02:34.848 ********** 2020-04-10T07:01:24.427 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => {"changed": false, "cmd": ["/usr/bin/pip2", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "", "stderr_lines": [], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages\nRequirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages", "Requirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2020-04-10T07:01:24.485 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:24 +0000 (0:00:00.086) 0:02:34.935 ********** 2020-04-10T07:01:24.913 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2020-04-10T07:01:24.914 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => {"changed": false} 2020-04-10T07:01:24.962 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:24 +0000 (0:00:00.479) 0:02:35.414 ********** 2020-04-10T07:01:24.993 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ ok: [smithi101.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp3s0f1"}, "changed": false} 2020-04-10T07:01:25.057 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:25 +0000 (0:00:00.091) 0:02:35.506 ********** 2020-04-10T07:01:25.061 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2020-04-10T07:01:25.067 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => {"changed": false} 2020-04-10T07:01:25.133 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:25 +0000 (0:00:00.080) 0:02:35.586 ********** 2020-04-10T07:01:25.163 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ 2020-04-10T07:01:25.193 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp3s0f1"}, "changed": false} 2020-04-10T07:01:25.205 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:25 +0000 (0:00:00.071) 0:02:35.658 ********** 2020-04-10T07:01:25.212 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** ok: [smithi101.front.sepia.ceph.com] => {"changed": false, "stat": {"atime": 1586501701.1125114, "attr_flags": "e", "attributes": ["extents"], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "ctime": 1586501699.6732311, "dev": 2049, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 110643, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/plain", "mode": "0644", "mtime": 1586501699.6732311, "nlink": 1, "path": "/etc/network/interfaces", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 71, "uid": 0, "version": "2999124757", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}} 2020-04-10T07:01:25.257 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:25 +0000 (0:00:00.051) 0:02:35.709 ********** 2020-04-10T07:01:25.383 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => {"changed": false, "stat": {"atime": 1586501687.8836765, "attr_flags": "e", "attributes": ["extents"], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "ctime": 1586501687.8716707, "dev": 2049, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 110643, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/plain", "mode": "0644", "mtime": 1586501687.8716707, "nlink": 1, "path": "/etc/network/interfaces", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 71, "uid": 0, "version": "2999124757", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}} 2020-04-10T07:01:25.437 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:25 +0000 (0:00:00.180) 0:02:35.890 ********** 2020-04-10T07:01:25.685 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** ok: [smithi204.front.sepia.ceph.com] => {"changed": false, "msg": ""} 2020-04-10T07:01:25.686 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => {"changed": false, "msg": ""} 2020-04-10T07:01:25.732 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:25 +0000 (0:00:00.292) 0:02:36.182 ********** 2020-04-10T07:01:25.771 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:25 +0000 (0:00:00.039) 0:02:36.222 ********** 2020-04-10T07:01:25.775 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2020-04-10T07:01:25.778 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2020-04-10T07:01:25.817 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:25 +0000 (0:00:00.047) 0:02:36.270 ********** 2020-04-10T07:01:25.823 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2020-04-10T07:01:25.877 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:25 +0000 (0:00:00.060) 0:02:36.330 ********** 2020-04-10T07:01:25.893 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:25.929 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:25 +0000 (0:00:00.051) 0:02:36.382 ********** 2020-04-10T07:01:25.935 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:25.983 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:25 +0000 (0:00:00.054) 0:02:36.436 ********** 2020-04-10T07:01:26.079 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** changed: [smithi101.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2020-04-10T07:01:26.139 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:26 +0000 (0:00:00.148) 0:02:36.584 ********** changed: [smithi204.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2020-04-10T07:01:26.186 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:26 +0000 (0:00:00.053) 0:02:36.638 ********** 2020-04-10T07:01:26.269 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* changed: [smithi101.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2020-04-10T07:01:26.318 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:26 +0000 (0:00:00.132) 0:02:36.771 ********** 2020-04-10T07:01:26.335 INFO:teuthology.task.ansible.out:changed: [smithi204.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2020-04-10T07:01:26.378 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:26 +0000 (0:00:00.060) 0:02:36.831 ********** 2020-04-10T07:01:26.384 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2020-04-10T07:01:26.386 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:26.426 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:26 +0000 (0:00:00.047) 0:02:36.879 ********** 2020-04-10T07:01:26.432 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:26.491 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:26 +0000 (0:00:00.065) 0:02:36.944 ********** 2020-04-10T07:01:26.499 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* skipping: [smithi101.front.sepia.ceph.com] => (item=enp3s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False"} 2020-04-10T07:01:26.502 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item=enp3s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False"} skipping: [smithi101.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} 2020-04-10T07:01:26.503 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} skipping: [smithi101.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2020-04-10T07:01:26.561 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:26 +0000 (0:00:00.069) 0:02:37.014 ********** 2020-04-10T07:01:26.566 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item=enp3s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False"} 2020-04-10T07:01:26.568 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2020-04-10T07:01:26.569 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} 2020-04-10T07:01:26.569 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2020-04-10T07:01:26.571 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item=enp3s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False"} 2020-04-10T07:01:26.634 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:26 +0000 (0:00:00.073) 0:02:37.087 ********** 2020-04-10T07:01:26.642 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** skipping: [smithi101.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'enp3s0f1', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:26.644 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'enp3s0f0', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:26.645 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'eno2', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:26.647 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'lo', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi101.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'eno1', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:26.700 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:26 +0000 (0:00:00.065) 0:02:37.153 ********** 2020-04-10T07:01:26.709 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'enp3s0f1', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:26.711 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'lo', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:26.713 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'eno2', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi204.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'eno1', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:26.714 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'enp3s0f0', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:26.755 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:26 +0000 (0:00:00.054) 0:02:37.208 ********** 2020-04-10T07:01:26.761 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-04-10T07:01:26.761 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:26.813 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:26 +0000 (0:00:00.057) 0:02:37.266 ********** 2020-04-10T07:01:26.825 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:26.865 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:26 +0000 (0:00:00.052) 0:02:37.318 ********** 2020-04-10T07:01:26.871 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2020-04-10T07:01:26.873 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:26.914 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:26 +0000 (0:00:00.048) 0:02:37.367 ********** 2020-04-10T07:01:26.923 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:26.962 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:26 +0000 (0:00:00.048) 0:02:37.415 ********** 2020-04-10T07:01:26.968 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2020-04-10T07:01:26.969 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:27.014 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:27 +0000 (0:00:00.050) 0:02:37.466 ********** 2020-04-10T07:01:27.019 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:27.067 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:27 +0000 (0:00:00.053) 0:02:37.520 ********** 2020-04-10T07:01:27.073 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2020-04-10T07:01:27.073 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:27.117 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:27 +0000 (0:00:00.050) 0:02:37.570 ********** 2020-04-10T07:01:27.124 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:27.165 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:27 +0000 (0:00:00.048) 0:02:37.618 ********** 2020-04-10T07:01:27.173 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:27.214 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:27 +0000 (0:00:00.048) 0:02:37.667 ********** 2020-04-10T07:01:27.222 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:27.263 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:27 +0000 (0:00:00.048) 0:02:37.716 ********** 2020-04-10T07:01:27.268 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-04-10T07:01:27.269 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:27.317 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:27 +0000 (0:00:00.054) 0:02:37.770 ********** 2020-04-10T07:01:27.329 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:27.368 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:27 +0000 (0:00:00.051) 0:02:37.821 ********** 2020-04-10T07:01:27.486 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** 2020-04-10T07:01:27.488 INFO:teuthology.task.ansible.out:skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:27.654 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:27 +0000 (0:00:00.285) 0:02:38.107 ********** 2020-04-10T07:01:27.661 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:27.744 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:27 +0000 (0:00:00.089) 0:02:38.197 ********** 2020-04-10T07:01:27.760 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** skipping: [smithi101.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:27.799 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:27 +0000 (0:00:00.055) 0:02:38.252 ********** 2020-04-10T07:01:27.811 INFO:teuthology.task.ansible.out:skipping: [smithi204.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-04-10T07:01:27.849 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:27 +0000 (0:00:00.049) 0:02:38.301 ********** 2020-04-10T07:01:27.975 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** ok: [smithi101.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.011079", "end": "2020-04-10 07:01:27.935999", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2020-04-10 07:01:27.924920", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-04-10T07:01:28.021 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.010607", "end": "2020-04-10 07:01:28.171773", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2020-04-10 07:01:28.161166", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-04-10T07:01:28.023 INFO:teuthology.task.ansible.out: 2020-04-10T07:01:28.130 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.005834", "end": "2020-04-10 07:01:28.077196", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2020-04-10 07:01:28.071362", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-04-10T07:01:28.168 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.004642", "end": "2020-04-10 07:01:28.315504", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2020-04-10 07:01:28.310862", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-04-10T07:01:28.267 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.005467", "end": "2020-04-10 07:01:28.231153", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2020-04-10 07:01:28.225686", "stderr": "find: '/etc/ceph': No such file or directory", "stderr_lines": ["find: '/etc/ceph': No such file or directory"], "stdout": "", "stdout_lines": []} 2020-04-10T07:01:28.319 INFO:teuthology.task.ansible.out:ok: [smithi204.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.005363", "end": "2020-04-10 07:01:28.439262", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2020-04-10 07:01:28.433899", "stderr": "find: '/etc/ceph': No such file or directory", "stderr_lines": ["find: '/etc/ceph': No such file or directory"], "stdout": "", "stdout_lines": []} 2020-04-10T07:01:28.441 INFO:teuthology.task.ansible.out:ok: [smithi101.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.037943", "end": "2020-04-10 07:01:28.416108", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2020-04-10 07:01:28.378165", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-04-10T07:01:28.530 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:28 +0000 (0:00:00.676) 0:02:38.978 ********** 2020-04-10T07:01:28.564 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2020-04-10T07:01:28.605 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:28 +0000 (0:00:00.079) 0:02:39.058 ********** 2020-04-10T07:01:28.613 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** ok: [smithi204.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.121444", "end": "2020-04-10 07:01:28.720593", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2020-04-10 07:01:28.599149", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-04-10T07:01:28.657 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:28 +0000 (0:00:00.052) 0:02:39.110 ********** 2020-04-10T07:01:28.701 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2020-04-10T07:01:28.754 INFO:teuthology.task.ansible.out:Friday 10 April 2020 07:01:28 +0000 (0:00:00.096) 0:02:39.207 ********** 2020-04-10T07:01:28.760 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2020-04-10T07:01:28.764 INFO:teuthology.task.ansible.out:changed: [smithi101.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "size": 0, "state": "file", "uid": 0} 2020-04-10T07:01:28.886 INFO:teuthology.task.ansible.out:changed: [smithi204.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "size": 0, "state": "file", "uid": 0} 2020-04-10T07:01:28.995 INFO:teuthology.task.ansible.out: PLAY [cobbler] ***************************************************************** 2020-04-10T07:01:28.997 INFO:teuthology.task.ansible.out:skipping: no hosts matched 2020-04-10T07:01:28.998 INFO:teuthology.task.ansible.out: PLAY [paddles] ***************************************************************** skipping: no hosts matched 2020-04-10T07:01:29.001 INFO:teuthology.task.ansible.out: PLAY [pulpito] ***************************************************************** skipping: no hosts matched 2020-04-10T07:01:29.002 INFO:teuthology.task.ansible.out: PLAY RECAP ********************************************************************* smithi101.front.sepia.ceph.com : ok=113 changed=33 unreachable=0 failed=0 skipped=170 rescued=0 ignored=0 smithi204.front.sepia.ceph.com : ok=113 changed=33 unreachable=0 failed=0 skipped=170 rescued=0 ignored=0 Friday 10 April 2020 07:01:28 +0000 (0:00:00.247) 0:02:39.454 ********** =============================================================================== 2020-04-10T07:01:29.004 INFO:teuthology.task.ansible.out:users : Create all admin users with sudo access. ----------------------- 17.84s users : Update authorized_keys using the keys repo --------------------- 15.84s testnode : Upgrade packages -------------------------------------------- 15.17s testnode : Ensure packages are not present. ----------------------------- 9.80s testnode : Install packages --------------------------------------------- 8.89s testnode : Zap all non-root disks --------------------------------------- 8.21s users : Remove revoked users -------------------------------------------- 5.28s Gathering Facts --------------------------------------------------------- 2.97s testnode : Update apt cache. -------------------------------------------- 2.70s common : Update apt cache ----------------------------------------------- 1.99s testnode : Install apt keys --------------------------------------------- 1.73s testnode : Create logical volume(s) ------------------------------------- 1.71s testnode : Blow away lingering OSD data and FSIDs ----------------------- 1.52s testnode : Install packages via pip ------------------------------------- 1.03s common : Set the hardware clock ----------------------------------------- 0.93s common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/. --- 0.84s ansible-managed : Create the cephlab_sudo sudoers.d file. --------------- 0.84s Gathering Facts --------------------------------------------------------- 0.83s testnode : Install python-pip on apt based systems. --------------------- 0.82s testnode : Ensure perl-doc is installed on apt systems. ----------------- 0.80s 2020-04-10T07:01:29.291 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi101.front.sepia.ceph.com'), Remote(name='ubuntu@smithi204.front.sepia.ceph.com')] 2020-04-10T07:01:29.294 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-04-10T07:01:29.295 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi101.front.sepia.ceph.com', 'timeout': 60} 2020-04-10T07:01:29.375 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:01:29.581 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-04-10T07:01:29.582 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi204.front.sepia.ceph.com', 'timeout': 60} 2020-04-10T07:01:29.671 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:01:29.861 INFO:teuthology.run_tasks:Running task clock... 2020-04-10T07:01:29.873 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2020-04-10T07:01:29.874 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:01:29.881 INFO:teuthology.orchestra.run:Running command with timeout 60 2020-04-10T07:01:29.884 INFO:teuthology.orchestra.run.smithi101:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2020-04-10T07:01:29.996 INFO:teuthology.orchestra.run.smithi101.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2020-04-10T07:01:29.996 INFO:teuthology.orchestra.run.smithi101.stderr:restrict ::: KOD does nothing without LIMITED. 2020-04-10T07:01:29.997 INFO:teuthology.orchestra.run.smithi101.stdout:10 Apr 07:01:29 ntpd[8229]: ntpd 4.2.8p10@1.3728-o (1): Starting 2020-04-10T07:01:29.997 INFO:teuthology.orchestra.run.smithi101.stdout:10 Apr 07:01:29 ntpd[8229]: Command line: ntpd -gq 2020-04-10T07:01:29.997 INFO:teuthology.orchestra.run.smithi101.stdout:10 Apr 07:01:29 ntpd[8229]: proto: precision = 0.120 usec (-23) 2020-04-10T07:01:29.997 INFO:teuthology.orchestra.run.smithi101.stdout:10 Apr 07:01:29 ntpd[8229]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2020-04-10T07:01:29.997 INFO:teuthology.orchestra.run.smithi101.stdout:10 Apr 07:01:29 ntpd[8229]: restrict ::: KOD does nothing without LIMITED. 2020-04-10T07:01:29.997 INFO:teuthology.orchestra.run.smithi101.stdout:10 Apr 07:01:29 ntpd[8229]: Listen and drop on 0 v6wildcard [::]:123 2020-04-10T07:01:29.998 INFO:teuthology.orchestra.run.smithi101.stdout:10 Apr 07:01:29 ntpd[8229]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2020-04-10T07:01:29.998 INFO:teuthology.orchestra.run.smithi101.stdout:10 Apr 07:01:29 ntpd[8229]: Listen normally on 2 lo 127.0.0.1:123 2020-04-10T07:01:29.998 INFO:teuthology.orchestra.run.smithi101.stdout:10 Apr 07:01:29 ntpd[8229]: Listen normally on 3 enp3s0f1 172.21.15.101:123 2020-04-10T07:01:29.998 INFO:teuthology.orchestra.run.smithi101.stdout:10 Apr 07:01:29 ntpd[8229]: Listen normally on 4 lo [::1]:123 2020-04-10T07:01:29.998 INFO:teuthology.orchestra.run.smithi101.stdout:10 Apr 07:01:29 ntpd[8229]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d723%5]:123 2020-04-10T07:01:29.998 INFO:teuthology.orchestra.run.smithi101.stdout:10 Apr 07:01:29 ntpd[8229]: Listening on routing socket on fd #22 for interface updates 2020-04-10T07:01:35.036 INFO:teuthology.orchestra.run.smithi101.stderr:10 Apr 07:01:34 ntpd[8229]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-04-10T07:01:35.036 INFO:teuthology.orchestra.run.smithi101.stderr:10 Apr 07:01:34 ntpd[8229]: can't open /var/log/ntpstats/rawstats.20200410: Permission denied 2020-04-10T07:01:35.037 INFO:teuthology.orchestra.run.smithi101.stderr:10 Apr 07:01:34 ntpd[8229]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-04-10T07:01:35.037 INFO:teuthology.orchestra.run.smithi101.stderr:10 Apr 07:01:34 ntpd[8229]: can't open /var/log/ntpstats/peerstats.20200410: Permission denied 2020-04-10T07:01:37.000 INFO:teuthology.orchestra.run.smithi101.stderr:10 Apr 07:01:36 ntpd[8229]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-04-10T07:01:37.000 INFO:teuthology.orchestra.run.smithi101.stderr:10 Apr 07:01:36 ntpd[8229]: can't open /var/log/ntpstats/rawstats.20200410: Permission denied 2020-04-10T07:01:37.000 INFO:teuthology.orchestra.run.smithi101.stderr:10 Apr 07:01:36 ntpd[8229]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-04-10T07:01:37.000 INFO:teuthology.orchestra.run.smithi101.stderr:10 Apr 07:01:36 ntpd[8229]: can't open /var/log/ntpstats/peerstats.20200410: Permission denied 2020-04-10T07:01:38.999 INFO:teuthology.orchestra.run.smithi101.stderr:10 Apr 07:01:38 ntpd[8229]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-04-10T07:01:38.999 INFO:teuthology.orchestra.run.smithi101.stderr:10 Apr 07:01:38 ntpd[8229]: can't open /var/log/ntpstats/rawstats.20200410: Permission denied 2020-04-10T07:01:39.000 INFO:teuthology.orchestra.run.smithi101.stderr:10 Apr 07:01:38 ntpd[8229]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-04-10T07:01:39.000 INFO:teuthology.orchestra.run.smithi101.stderr:10 Apr 07:01:38 ntpd[8229]: can't open /var/log/ntpstats/peerstats.20200410: Permission denied 2020-04-10T07:01:40.996 INFO:teuthology.orchestra.run.smithi101.stdout:10 Apr 07:01:40 ntpd[8229]: ntpd: time slew -0.005332 s 2020-04-10T07:01:40.996 INFO:teuthology.orchestra.run.smithi101.stdout:ntpd: time slew -0.005332s 2020-04-10T07:01:40.997 INFO:teuthology.orchestra.run.smithi101.stderr:10 Apr 07:01:40 ntpd[8229]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-04-10T07:01:40.997 INFO:teuthology.orchestra.run.smithi101.stderr:10 Apr 07:01:40 ntpd[8229]: can't open /var/log/ntpstats/rawstats.20200410: Permission denied 2020-04-10T07:01:40.997 INFO:teuthology.orchestra.run.smithi101.stderr:10 Apr 07:01:40 ntpd[8229]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-04-10T07:01:40.997 INFO:teuthology.orchestra.run.smithi101.stderr:10 Apr 07:01:40 ntpd[8229]: can't open /var/log/ntpstats/peerstats.20200410: Permission denied 2020-04-10T07:01:40.997 INFO:teuthology.orchestra.run.smithi101.stderr:10 Apr 07:01:40 ntpd[8229]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2020-04-10T07:01:40.997 INFO:teuthology.orchestra.run.smithi101.stderr:10 Apr 07:01:40 ntpd[8229]: can't open /var/log/ntpstats/loopstats.20200410: Permission denied 2020-04-10T07:01:41.070 INFO:teuthology.orchestra.run.smithi101.stdout: remote refid st t when poll reach delay offset jitter 2020-04-10T07:01:41.070 INFO:teuthology.orchestra.run.smithi101.stdout:============================================================================== 2020-04-10T07:01:41.070 INFO:teuthology.orchestra.run.smithi101.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-04-10T07:01:41.070 INFO:teuthology.orchestra.run.smithi101.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-04-10T07:01:41.070 INFO:teuthology.orchestra.run.smithi101.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-04-10T07:01:41.070 INFO:teuthology.orchestra.run.smithi101.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-04-10T07:01:41.071 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:01:41.077 INFO:teuthology.orchestra.run:Running command with timeout 60 2020-04-10T07:01:41.077 INFO:teuthology.orchestra.run.smithi204:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2020-04-10T07:01:41.195 INFO:teuthology.orchestra.run.smithi204.stdout:10 Apr 07:01:41 ntpd[8085]: ntpd 4.2.8p10@1.3728-o (1): Starting 2020-04-10T07:01:41.196 INFO:teuthology.orchestra.run.smithi204.stdout:10 Apr 07:01:41 ntpd[8085]: Command line: ntpd -gq 2020-04-10T07:01:41.196 INFO:teuthology.orchestra.run.smithi204.stdout:10 Apr 07:01:41 ntpd[8085]: proto: precision = 0.109 usec (-23) 2020-04-10T07:01:41.200 INFO:teuthology.orchestra.run.smithi204.stdout:10 Apr 07:01:41 ntpd[8085]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2020-04-10T07:01:41.200 INFO:teuthology.orchestra.run.smithi204.stdout:10 Apr 07:01:41 ntpd[8085]: restrict ::: KOD does nothing without LIMITED. 2020-04-10T07:01:41.200 INFO:teuthology.orchestra.run.smithi204.stdout:10 Apr 07:01:41 ntpd[8085]: Listen and drop on 0 v6wildcard [::]:123 2020-04-10T07:01:41.201 INFO:teuthology.orchestra.run.smithi204.stdout:10 Apr 07:01:41 ntpd[8085]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2020-04-10T07:01:41.201 INFO:teuthology.orchestra.run.smithi204.stdout:10 Apr 07:01:41 ntpd[8085]: Listen normally on 2 lo 127.0.0.1:123 2020-04-10T07:01:41.201 INFO:teuthology.orchestra.run.smithi204.stdout:10 Apr 07:01:41 ntpd[8085]: Listen normally on 3 enp3s0f1 172.21.15.204:123 2020-04-10T07:01:41.201 INFO:teuthology.orchestra.run.smithi204.stdout:10 Apr 07:01:41 ntpd[8085]: Listen normally on 4 lo [::1]:123 2020-04-10T07:01:41.201 INFO:teuthology.orchestra.run.smithi204.stdout:10 Apr 07:01:41 ntpd[8085]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:77ab%5]:123 2020-04-10T07:01:41.201 INFO:teuthology.orchestra.run.smithi204.stdout:10 Apr 07:01:41 ntpd[8085]: Listening on routing socket on fd #22 for interface updates 2020-04-10T07:01:41.246 INFO:teuthology.orchestra.run.smithi204.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2020-04-10T07:01:41.246 INFO:teuthology.orchestra.run.smithi204.stderr:restrict ::: KOD does nothing without LIMITED. 2020-04-10T07:01:46.242 INFO:teuthology.orchestra.run.smithi204.stderr:10 Apr 07:01:46 ntpd[8085]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-04-10T07:01:46.242 INFO:teuthology.orchestra.run.smithi204.stderr:10 Apr 07:01:46 ntpd[8085]: can't open /var/log/ntpstats/rawstats.20200410: Permission denied 2020-04-10T07:01:46.242 INFO:teuthology.orchestra.run.smithi204.stderr:10 Apr 07:01:46 ntpd[8085]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-04-10T07:01:46.243 INFO:teuthology.orchestra.run.smithi204.stderr:10 Apr 07:01:46 ntpd[8085]: can't open /var/log/ntpstats/peerstats.20200410: Permission denied 2020-04-10T07:01:48.209 INFO:teuthology.orchestra.run.smithi204.stderr:10 Apr 07:01:48 ntpd[8085]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-04-10T07:01:48.209 INFO:teuthology.orchestra.run.smithi204.stderr:10 Apr 07:01:48 ntpd[8085]: can't open /var/log/ntpstats/rawstats.20200410: Permission denied 2020-04-10T07:01:48.209 INFO:teuthology.orchestra.run.smithi204.stderr:10 Apr 07:01:48 ntpd[8085]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-04-10T07:01:48.209 INFO:teuthology.orchestra.run.smithi204.stderr:10 Apr 07:01:48 ntpd[8085]: can't open /var/log/ntpstats/peerstats.20200410: Permission denied 2020-04-10T07:01:50.207 INFO:teuthology.orchestra.run.smithi204.stderr:10 Apr 07:01:50 ntpd[8085]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-04-10T07:01:50.208 INFO:teuthology.orchestra.run.smithi204.stderr:10 Apr 07:01:50 ntpd[8085]: can't open /var/log/ntpstats/rawstats.20200410: Permission denied 2020-04-10T07:01:50.208 INFO:teuthology.orchestra.run.smithi204.stderr:10 Apr 07:01:50 ntpd[8085]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-04-10T07:01:50.208 INFO:teuthology.orchestra.run.smithi204.stderr:10 Apr 07:01:50 ntpd[8085]: can't open /var/log/ntpstats/peerstats.20200410: Permission denied 2020-04-10T07:01:52.240 INFO:teuthology.orchestra.run.smithi204.stdout:10 Apr 07:01:52 ntpd[8085]: ntpd: time set -0.172477 s 2020-04-10T07:01:52.240 INFO:teuthology.orchestra.run.smithi204.stdout:ntpd: time set -0.172477s 2020-04-10T07:01:52.249 INFO:teuthology.orchestra.run.smithi204.stderr:10 Apr 07:01:52 ntpd[8085]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-04-10T07:01:52.249 INFO:teuthology.orchestra.run.smithi204.stderr:10 Apr 07:01:52 ntpd[8085]: can't open /var/log/ntpstats/rawstats.20200410: Permission denied 2020-04-10T07:01:52.249 INFO:teuthology.orchestra.run.smithi204.stderr:10 Apr 07:01:52 ntpd[8085]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-04-10T07:01:52.249 INFO:teuthology.orchestra.run.smithi204.stderr:10 Apr 07:01:52 ntpd[8085]: can't open /var/log/ntpstats/peerstats.20200410: Permission denied 2020-04-10T07:01:52.250 INFO:teuthology.orchestra.run.smithi204.stderr:10 Apr 07:01:52 ntpd[8085]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2020-04-10T07:01:52.250 INFO:teuthology.orchestra.run.smithi204.stderr:10 Apr 07:01:52 ntpd[8085]: can't open /var/log/ntpstats/loopstats.20200410: Permission denied 2020-04-10T07:01:52.460 INFO:teuthology.orchestra.run.smithi204.stdout: remote refid st t when poll reach delay offset jitter 2020-04-10T07:01:52.460 INFO:teuthology.orchestra.run.smithi204.stdout:============================================================================== 2020-04-10T07:01:52.460 INFO:teuthology.orchestra.run.smithi204.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-04-10T07:01:52.460 INFO:teuthology.orchestra.run.smithi204.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-04-10T07:01:52.461 INFO:teuthology.orchestra.run.smithi204.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-04-10T07:01:52.461 INFO:teuthology.orchestra.run.smithi204.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-04-10T07:01:52.467 INFO:teuthology.run_tasks:Running task install... 2020-04-10T07:01:52.478 DEBUG:teuthology.task.install:project ceph 2020-04-10T07:01:52.478 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'sha1': '39357d2a782a02d1ea09c76cbe7bc6f222a1127a'}} 2020-04-10T07:01:52.479 DEBUG:teuthology.task.install:config {'sha1': '39357d2a782a02d1ea09c76cbe7bc6f222a1127a', 'extra_packages': ['rbd-mirror']} 2020-04-10T07:01:52.479 INFO:teuthology.task.install:Using flavor: basic 2020-04-10T07:01:52.488 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2020-04-10T07:01:52.488 INFO:teuthology.task.install:extra packages: ['rbd-mirror'] 2020-04-10T07:01:52.488 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:01:52.494 INFO:teuthology.orchestra.run.smithi101:> sudo lsb_release -is 2020-04-10T07:01:52.588 INFO:teuthology.orchestra.run.smithi101.stdout:Ubuntu 2020-04-10T07:01:52.592 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-04-10T07:01:52.593 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:01:52.593 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:01:52.598 INFO:teuthology.orchestra.run.smithi204:> sudo lsb_release -is 2020-04-10T07:01:52.599 INFO:teuthology.orchestra.run.smithi101:> sudo apt-key list | grep Ceph 2020-04-10T07:01:52.694 INFO:teuthology.orchestra.run.smithi204.stdout:Ubuntu 2020-04-10T07:01:52.697 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-04-10T07:01:52.697 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:01:52.705 INFO:teuthology.orchestra.run.smithi204:> sudo apt-key list | grep Ceph 2020-04-10T07:01:52.733 INFO:teuthology.orchestra.run.smithi101.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2020-04-10T07:01:52.791 INFO:teuthology.orchestra.run.smithi101.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2020-04-10T07:01:52.792 INFO:teuthology.orchestra.run.smithi101.stdout:uid [ unknown] Ceph.com (release key) 2020-04-10T07:01:52.793 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:01:52.798 INFO:teuthology.orchestra.run.smithi101:> sudo lsb_release -is 2020-04-10T07:01:52.845 INFO:teuthology.orchestra.run.smithi204.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2020-04-10T07:01:52.889 INFO:teuthology.orchestra.run.smithi101.stdout:Ubuntu 2020-04-10T07:01:52.894 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-04-10T07:01:52.895 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, rbd-mirror on remote deb x86_64 2020-04-10T07:01:52.895 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=39357d2a782a02d1ea09c76cbe7bc6f222a1127a 2020-04-10T07:01:52.900 INFO:teuthology.orchestra.run.smithi204.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2020-04-10T07:01:52.900 INFO:teuthology.orchestra.run.smithi204.stdout:uid [ unknown] Ceph.com (release key) 2020-04-10T07:01:52.901 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:01:52.906 INFO:teuthology.orchestra.run.smithi204:> sudo lsb_release -is 2020-04-10T07:01:53.002 INFO:teuthology.orchestra.run.smithi204.stdout:Ubuntu 2020-04-10T07:01:53.007 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-04-10T07:01:53.007 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, rbd-mirror on remote deb x86_64 2020-04-10T07:01:53.008 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=39357d2a782a02d1ea09c76cbe7bc6f222a1127a 2020-04-10T07:01:53.131 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default/ 2020-04-10T07:01:53.131 INFO:teuthology.task.install.deb:Package version is 15.2.0-148-g39357d2-1bionic 2020-04-10T07:01:53.243 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default/ 2020-04-10T07:01:53.243 INFO:teuthology.task.install.deb:Package version is 15.2.0-148-g39357d2-1bionic 2020-04-10T07:01:53.321 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:01:53.327 INFO:teuthology.orchestra.run.smithi101:> sudo sh -c 'cat > /etc/apt/sources.list.d/ceph.list' 2020-04-10T07:01:53.414 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:01:53.435 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:01:53.443 INFO:teuthology.orchestra.run.smithi204:> sudo sh -c 'cat > /etc/apt/sources.list.d/ceph.list' 2020-04-10T07:01:53.461 INFO:teuthology.orchestra.run.smithi101:> sudo apt-get update 2020-04-10T07:01:53.531 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:01:53.577 INFO:teuthology.orchestra.run.smithi204:> sudo apt-get update 2020-04-10T07:01:53.761 INFO:teuthology.orchestra.run.smithi101.stdout:Get:1 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB] 2020-04-10T07:01:53.768 INFO:teuthology.orchestra.run.smithi204.stdout:Hit:1 http://security.ubuntu.com/ubuntu bionic-security InRelease 2020-04-10T07:01:53.774 INFO:teuthology.orchestra.run.smithi101.stdout:Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease 2020-04-10T07:01:53.797 INFO:teuthology.orchestra.run.smithi101.stdout:Ign:3 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic InRelease 2020-04-10T07:01:53.836 INFO:teuthology.orchestra.run.smithi101.stdout:Get:4 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic Release [7,672 B] 2020-04-10T07:01:53.842 INFO:teuthology.orchestra.run.smithi101.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB] 2020-04-10T07:01:53.872 INFO:teuthology.orchestra.run.smithi101.stdout:Ign:6 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic Release.gpg 2020-04-10T07:01:53.881 INFO:teuthology.orchestra.run.smithi204.stdout:Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease 2020-04-10T07:01:53.906 INFO:teuthology.orchestra.run.smithi204.stdout:Ign:3 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic InRelease 2020-04-10T07:01:53.917 INFO:teuthology.orchestra.run.smithi101.stdout:Get:7 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main i386 Packages [3,069 B] 2020-04-10T07:01:53.944 INFO:teuthology.orchestra.run.smithi204.stdout:Get:4 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic Release [7,672 B] 2020-04-10T07:01:53.962 INFO:teuthology.orchestra.run.smithi204.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB] 2020-04-10T07:01:53.977 INFO:teuthology.orchestra.run.smithi101.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main amd64 Packages [15.8 kB] 2020-04-10T07:01:53.982 INFO:teuthology.orchestra.run.smithi204.stdout:Ign:6 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic Release.gpg 2020-04-10T07:01:54.019 INFO:teuthology.orchestra.run.smithi204.stdout:Get:7 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main i386 Packages [3,069 B] 2020-04-10T07:01:54.084 INFO:teuthology.orchestra.run.smithi204.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main amd64 Packages [15.8 kB] 2020-04-10T07:01:54.167 INFO:teuthology.orchestra.run.smithi101.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB] 2020-04-10T07:01:54.290 INFO:teuthology.orchestra.run.smithi204.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB] 2020-04-10T07:01:54.487 INFO:teuthology.orchestra.run.smithi101.stdout:Fetched 279 kB in 1s (338 kB/s) 2020-04-10T07:01:54.540 INFO:teuthology.orchestra.run.smithi204.stdout:Fetched 190 kB in 1s (251 kB/s) 2020-04-10T07:01:55.650 INFO:teuthology.orchestra.run.smithi101.stdout:Reading package lists... 2020-04-10T07:01:55.692 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:01:55.699 INFO:teuthology.orchestra.run.smithi101:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=15.2.0-148-g39357d2-1bionic cephadm=15.2.0-148-g39357d2-1bionic ceph-mds=15.2.0-148-g39357d2-1bionic ceph-mgr=15.2.0-148-g39357d2-1bionic ceph-common=15.2.0-148-g39357d2-1bionic ceph-fuse=15.2.0-148-g39357d2-1bionic ceph-test=15.2.0-148-g39357d2-1bionic radosgw=15.2.0-148-g39357d2-1bionic python3-rados=15.2.0-148-g39357d2-1bionic python3-rgw=15.2.0-148-g39357d2-1bionic python3-cephfs=15.2.0-148-g39357d2-1bionic python3-rbd=15.2.0-148-g39357d2-1bionic libcephfs2=15.2.0-148-g39357d2-1bionic libcephfs-dev=15.2.0-148-g39357d2-1bionic librados2=15.2.0-148-g39357d2-1bionic librbd1=15.2.0-148-g39357d2-1bionic rbd-fuse=15.2.0-148-g39357d2-1bionic rbd-mirror=15.2.0-148-g39357d2-1bionic 2020-04-10T07:01:55.707 INFO:teuthology.orchestra.run.smithi204.stdout:Reading package lists... 2020-04-10T07:01:55.755 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:01:55.765 INFO:teuthology.orchestra.run.smithi204:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=15.2.0-148-g39357d2-1bionic cephadm=15.2.0-148-g39357d2-1bionic ceph-mds=15.2.0-148-g39357d2-1bionic ceph-mgr=15.2.0-148-g39357d2-1bionic ceph-common=15.2.0-148-g39357d2-1bionic ceph-fuse=15.2.0-148-g39357d2-1bionic ceph-test=15.2.0-148-g39357d2-1bionic radosgw=15.2.0-148-g39357d2-1bionic python3-rados=15.2.0-148-g39357d2-1bionic python3-rgw=15.2.0-148-g39357d2-1bionic python3-cephfs=15.2.0-148-g39357d2-1bionic python3-rbd=15.2.0-148-g39357d2-1bionic libcephfs2=15.2.0-148-g39357d2-1bionic libcephfs-dev=15.2.0-148-g39357d2-1bionic librados2=15.2.0-148-g39357d2-1bionic librbd1=15.2.0-148-g39357d2-1bionic rbd-fuse=15.2.0-148-g39357d2-1bionic rbd-mirror=15.2.0-148-g39357d2-1bionic 2020-04-10T07:01:55.821 INFO:teuthology.orchestra.run.smithi101.stdout:Reading package lists... 2020-04-10T07:01:55.895 INFO:teuthology.orchestra.run.smithi204.stdout:Reading package lists... 2020-04-10T07:01:56.073 INFO:teuthology.orchestra.run.smithi101.stdout:Building dependency tree... 2020-04-10T07:01:56.074 INFO:teuthology.orchestra.run.smithi101.stdout:Reading state information... 2020-04-10T07:01:56.142 INFO:teuthology.orchestra.run.smithi204.stdout:Building dependency tree... 2020-04-10T07:01:56.143 INFO:teuthology.orchestra.run.smithi204.stdout:Reading state information... 2020-04-10T07:01:56.233 INFO:teuthology.orchestra.run.smithi101.stdout:The following additional packages will be installed: 2020-04-10T07:01:56.234 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2020-04-10T07:01:56.234 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2020-04-10T07:01:56.235 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mgr-rook ceph-mon ceph-osd jq libjq1 libleveldb1v5 liblttng-ust0 2020-04-10T07:01:56.235 INFO:teuthology.orchestra.run.smithi101.stdout: liboath0 libonig4 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2020-04-10T07:01:56.235 INFO:teuthology.orchestra.run.smithi101.stdout: libsnappy1v5 libxslt1.1 python-attr python-funcsigs python-pastedeploy-tpl 2020-04-10T07:01:56.235 INFO:teuthology.orchestra.run.smithi101.stdout: python-pluggy python-py python-pytest python3-asn1crypto python3-attr 2020-04-10T07:01:56.235 INFO:teuthology.orchestra.run.smithi101.stdout: python3-bcrypt python3-bs4 python3-ceph-argparse python3-ceph-common 2020-04-10T07:01:56.235 INFO:teuthology.orchestra.run.smithi101.stdout: python3-cffi-backend python3-cherrypy3 python3-cryptography python3-cycler 2020-04-10T07:01:56.236 INFO:teuthology.orchestra.run.smithi101.stdout: python3-dateutil python3-decorator python3-fasteners python3-html5lib 2020-04-10T07:01:56.236 INFO:teuthology.orchestra.run.smithi101.stdout: python3-httplib2 python3-joblib python3-json-pointer python3-jsonpatch 2020-04-10T07:01:56.236 INFO:teuthology.orchestra.run.smithi101.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2020-04-10T07:01:56.236 INFO:teuthology.orchestra.run.smithi101.stdout: python3-markupsafe python3-matplotlib python3-monotonic python3-numpy 2020-04-10T07:01:56.236 INFO:teuthology.orchestra.run.smithi101.stdout: python3-oauth2client python3-olefile python3-openssl python3-paste 2020-04-10T07:01:56.236 INFO:teuthology.orchestra.run.smithi101.stdout: python3-pastedeploy python3-pastescript python3-pecan python3-pil 2020-04-10T07:01:56.236 INFO:teuthology.orchestra.run.smithi101.stdout: python3-pluggy python3-prettytable python3-py python3-pyasn1 2020-04-10T07:01:56.237 INFO:teuthology.orchestra.run.smithi101.stdout: python3-pyasn1-modules python3-pyinotify python3-pyparsing python3-pytest 2020-04-10T07:01:56.237 INFO:teuthology.orchestra.run.smithi101.stdout: python3-repoze.lru python3-routes python3-rsa python3-scipy 2020-04-10T07:01:56.237 INFO:teuthology.orchestra.run.smithi101.stdout: python3-setuptools python3-simplegeneric python3-simplejson 2020-04-10T07:01:56.237 INFO:teuthology.orchestra.run.smithi101.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib 2020-04-10T07:01:56.237 INFO:teuthology.orchestra.run.smithi101.stdout: python3-sqlalchemy python3-tempita python3-tk python3-tz python3-uritemplate 2020-04-10T07:01:56.237 INFO:teuthology.orchestra.run.smithi101.stdout: python3-waitress python3-webencodings python3-webob python3-websocket 2020-04-10T07:01:56.237 INFO:teuthology.orchestra.run.smithi101.stdout: python3-webtest python3-werkzeug socat xmlstarlet 2020-04-10T07:01:56.238 INFO:teuthology.orchestra.run.smithi101.stdout:Suggested packages: 2020-04-10T07:01:56.238 INFO:teuthology.orchestra.run.smithi101.stdout: python3-influxdb python-attr-doc python-funcsigs-doc subversion 2020-04-10T07:01:56.238 INFO:teuthology.orchestra.run.smithi101.stdout: python-pytest-xdist python-mock python-cryptography-doc 2020-04-10T07:01:56.238 INFO:teuthology.orchestra.run.smithi101.stdout: python3-cryptography-vectors python-cycler-doc python3-genshi python3-crypto 2020-04-10T07:01:56.238 INFO:teuthology.orchestra.run.smithi101.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc dvipng 2020-04-10T07:01:56.238 INFO:teuthology.orchestra.run.smithi101.stdout: ffmpeg gir1.2-gtk-3.0 ghostscript inkscape ipython3 python-matplotlib-doc 2020-04-10T07:01:56.239 INFO:teuthology.orchestra.run.smithi101.stdout: python3-cairocffi python3-gi-cairo python3-gobject python3-pyqt4 python3-sip 2020-04-10T07:01:56.239 INFO:teuthology.orchestra.run.smithi101.stdout: python3-tornado texlive-extra-utils texlive-latex-extra ttf-staypuft 2020-04-10T07:01:56.239 INFO:teuthology.orchestra.run.smithi101.stdout: python-numpy-doc python3-dev python3-numpy-dbg python-openssl-doc 2020-04-10T07:01:56.239 INFO:teuthology.orchestra.run.smithi101.stdout: python3-openssl-dbg httpd-wsgi libapache2-mod-python libapache2-mod-scgi 2020-04-10T07:01:56.239 INFO:teuthology.orchestra.run.smithi101.stdout: libjs-mochikit python-pecan-doc python-pil-doc python3-pil-dbg 2020-04-10T07:01:56.239 INFO:teuthology.orchestra.run.smithi101.stdout: python-pyinotify-doc python-pyparsing-doc python-scipy-doc 2020-04-10T07:01:56.239 INFO:teuthology.orchestra.run.smithi101.stdout: python-setuptools-doc python3-dap python-sklearn-doc python-sqlalchemy-doc 2020-04-10T07:01:56.240 INFO:teuthology.orchestra.run.smithi101.stdout: python3-psycopg2 python3-pymysql python3-fdb tix python3-tk-dbg 2020-04-10T07:01:56.240 INFO:teuthology.orchestra.run.smithi101.stdout: python-waitress-doc python-webob-doc python-webtest-doc python3-termcolor 2020-04-10T07:01:56.240 INFO:teuthology.orchestra.run.smithi101.stdout: python3-watchdog python-werkzeug-doc 2020-04-10T07:01:56.302 INFO:teuthology.orchestra.run.smithi204.stdout:The following additional packages will be installed: 2020-04-10T07:01:56.302 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2020-04-10T07:01:56.302 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2020-04-10T07:01:56.302 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-rook ceph-mon ceph-osd jq libjq1 libleveldb1v5 liblttng-ust0 2020-04-10T07:01:56.302 INFO:teuthology.orchestra.run.smithi204.stdout: liboath0 libonig4 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2020-04-10T07:01:56.303 INFO:teuthology.orchestra.run.smithi204.stdout: libsnappy1v5 libxslt1.1 python-attr python-funcsigs python-pastedeploy-tpl 2020-04-10T07:01:56.304 INFO:teuthology.orchestra.run.smithi204.stdout: python-pluggy python-py python-pytest python3-asn1crypto python3-attr 2020-04-10T07:01:56.304 INFO:teuthology.orchestra.run.smithi204.stdout: python3-bcrypt python3-bs4 python3-ceph-argparse python3-ceph-common 2020-04-10T07:01:56.305 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cffi-backend python3-cherrypy3 python3-cryptography python3-cycler 2020-04-10T07:01:56.305 INFO:teuthology.orchestra.run.smithi204.stdout: python3-dateutil python3-decorator python3-fasteners python3-html5lib 2020-04-10T07:01:56.305 INFO:teuthology.orchestra.run.smithi204.stdout: python3-httplib2 python3-joblib python3-json-pointer python3-jsonpatch 2020-04-10T07:01:56.305 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2020-04-10T07:01:56.305 INFO:teuthology.orchestra.run.smithi204.stdout: python3-markupsafe python3-matplotlib python3-monotonic python3-numpy 2020-04-10T07:01:56.305 INFO:teuthology.orchestra.run.smithi204.stdout: python3-oauth2client python3-olefile python3-openssl python3-paste 2020-04-10T07:01:56.305 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pastedeploy python3-pastescript python3-pecan python3-pil 2020-04-10T07:01:56.306 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pluggy python3-prettytable python3-py python3-pyasn1 2020-04-10T07:01:56.306 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1-modules python3-pyinotify python3-pyparsing python3-pytest 2020-04-10T07:01:56.306 INFO:teuthology.orchestra.run.smithi204.stdout: python3-repoze.lru python3-routes python3-rsa python3-scipy 2020-04-10T07:01:56.306 INFO:teuthology.orchestra.run.smithi204.stdout: python3-setuptools python3-simplegeneric python3-simplejson 2020-04-10T07:01:56.306 INFO:teuthology.orchestra.run.smithi204.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib 2020-04-10T07:01:56.306 INFO:teuthology.orchestra.run.smithi204.stdout: python3-sqlalchemy python3-tempita python3-tk python3-tz python3-uritemplate 2020-04-10T07:01:56.306 INFO:teuthology.orchestra.run.smithi204.stdout: python3-waitress python3-webencodings python3-webob python3-websocket 2020-04-10T07:01:56.307 INFO:teuthology.orchestra.run.smithi204.stdout: python3-webtest python3-werkzeug socat xmlstarlet 2020-04-10T07:01:56.307 INFO:teuthology.orchestra.run.smithi204.stdout:Suggested packages: 2020-04-10T07:01:56.307 INFO:teuthology.orchestra.run.smithi204.stdout: python3-influxdb python-attr-doc python-funcsigs-doc subversion 2020-04-10T07:01:56.307 INFO:teuthology.orchestra.run.smithi204.stdout: python-pytest-xdist python-mock python-cryptography-doc 2020-04-10T07:01:56.307 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cryptography-vectors python-cycler-doc python3-genshi python3-crypto 2020-04-10T07:01:56.307 INFO:teuthology.orchestra.run.smithi204.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc dvipng 2020-04-10T07:01:56.307 INFO:teuthology.orchestra.run.smithi204.stdout: ffmpeg gir1.2-gtk-3.0 ghostscript inkscape ipython3 python-matplotlib-doc 2020-04-10T07:01:56.308 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cairocffi python3-gi-cairo python3-gobject python3-pyqt4 python3-sip 2020-04-10T07:01:56.308 INFO:teuthology.orchestra.run.smithi204.stdout: python3-tornado texlive-extra-utils texlive-latex-extra ttf-staypuft 2020-04-10T07:01:56.308 INFO:teuthology.orchestra.run.smithi204.stdout: python-numpy-doc python3-dev python3-numpy-dbg python-openssl-doc 2020-04-10T07:01:56.308 INFO:teuthology.orchestra.run.smithi204.stdout: python3-openssl-dbg httpd-wsgi libapache2-mod-python libapache2-mod-scgi 2020-04-10T07:01:56.308 INFO:teuthology.orchestra.run.smithi204.stdout: libjs-mochikit python-pecan-doc python-pil-doc python3-pil-dbg 2020-04-10T07:01:56.308 INFO:teuthology.orchestra.run.smithi204.stdout: python-pyinotify-doc python-pyparsing-doc python-scipy-doc 2020-04-10T07:01:56.308 INFO:teuthology.orchestra.run.smithi204.stdout: python-setuptools-doc python3-dap python-sklearn-doc python-sqlalchemy-doc 2020-04-10T07:01:56.309 INFO:teuthology.orchestra.run.smithi204.stdout: python3-psycopg2 python3-pymysql python3-fdb tix python3-tk-dbg 2020-04-10T07:01:56.309 INFO:teuthology.orchestra.run.smithi204.stdout: python-waitress-doc python-webob-doc python-webtest-doc python3-termcolor 2020-04-10T07:01:56.309 INFO:teuthology.orchestra.run.smithi204.stdout: python3-watchdog python-werkzeug-doc 2020-04-10T07:01:56.335 INFO:teuthology.orchestra.run.smithi101.stdout:The following NEW packages will be installed: 2020-04-10T07:01:56.335 INFO:teuthology.orchestra.run.smithi101.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2020-04-10T07:01:56.335 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2020-04-10T07:01:56.335 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2020-04-10T07:01:56.336 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mgr-rook ceph-mon ceph-osd ceph-test cephadm jq libcephfs-dev 2020-04-10T07:01:56.337 INFO:teuthology.orchestra.run.smithi101.stdout: libcephfs2 libjq1 libleveldb1v5 liblttng-ust0 liboath0 libonig4 librabbitmq4 2020-04-10T07:01:56.337 INFO:teuthology.orchestra.run.smithi101.stdout: libradosstriper1 librdkafka1 librgw2 libsnappy1v5 libxslt1.1 python-attr 2020-04-10T07:01:56.337 INFO:teuthology.orchestra.run.smithi101.stdout: python-funcsigs python-pastedeploy-tpl python-pluggy python-py python-pytest 2020-04-10T07:01:56.337 INFO:teuthology.orchestra.run.smithi101.stdout: python3-asn1crypto python3-attr python3-bcrypt python3-bs4 2020-04-10T07:01:56.337 INFO:teuthology.orchestra.run.smithi101.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs 2020-04-10T07:01:56.337 INFO:teuthology.orchestra.run.smithi101.stdout: python3-cffi-backend python3-cherrypy3 python3-cryptography python3-cycler 2020-04-10T07:01:56.337 INFO:teuthology.orchestra.run.smithi101.stdout: python3-dateutil python3-decorator python3-fasteners python3-html5lib 2020-04-10T07:01:56.338 INFO:teuthology.orchestra.run.smithi101.stdout: python3-httplib2 python3-joblib python3-json-pointer python3-jsonpatch 2020-04-10T07:01:56.338 INFO:teuthology.orchestra.run.smithi101.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2020-04-10T07:01:56.338 INFO:teuthology.orchestra.run.smithi101.stdout: python3-markupsafe python3-matplotlib python3-monotonic python3-numpy 2020-04-10T07:01:56.338 INFO:teuthology.orchestra.run.smithi101.stdout: python3-oauth2client python3-olefile python3-openssl python3-paste 2020-04-10T07:01:56.338 INFO:teuthology.orchestra.run.smithi101.stdout: python3-pastedeploy python3-pastescript python3-pecan python3-pil 2020-04-10T07:01:56.338 INFO:teuthology.orchestra.run.smithi101.stdout: python3-pluggy python3-prettytable python3-py python3-pyasn1 2020-04-10T07:01:56.338 INFO:teuthology.orchestra.run.smithi101.stdout: python3-pyasn1-modules python3-pyinotify python3-pyparsing python3-pytest 2020-04-10T07:01:56.339 INFO:teuthology.orchestra.run.smithi101.stdout: python3-rados python3-rbd python3-repoze.lru python3-rgw python3-routes 2020-04-10T07:01:56.341 INFO:teuthology.orchestra.run.smithi101.stdout: python3-rsa python3-scipy python3-setuptools python3-simplegeneric 2020-04-10T07:01:56.341 INFO:teuthology.orchestra.run.smithi101.stdout: python3-simplejson python3-singledispatch python3-sklearn 2020-04-10T07:01:56.341 INFO:teuthology.orchestra.run.smithi101.stdout: python3-sklearn-lib python3-sqlalchemy python3-tempita python3-tk python3-tz 2020-04-10T07:01:56.341 INFO:teuthology.orchestra.run.smithi101.stdout: python3-uritemplate python3-waitress python3-webencodings python3-webob 2020-04-10T07:01:56.342 INFO:teuthology.orchestra.run.smithi101.stdout: python3-websocket python3-webtest python3-werkzeug radosgw rbd-fuse 2020-04-10T07:01:56.342 INFO:teuthology.orchestra.run.smithi101.stdout: rbd-mirror socat xmlstarlet 2020-04-10T07:01:56.342 INFO:teuthology.orchestra.run.smithi101.stdout:The following packages will be upgraded: 2020-04-10T07:01:56.342 INFO:teuthology.orchestra.run.smithi101.stdout: librados2 librbd1 2020-04-10T07:01:56.404 INFO:teuthology.orchestra.run.smithi204.stdout:The following NEW packages will be installed: 2020-04-10T07:01:56.404 INFO:teuthology.orchestra.run.smithi204.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2020-04-10T07:01:56.404 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2020-04-10T07:01:56.404 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2020-04-10T07:01:56.405 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-rook ceph-mon ceph-osd ceph-test cephadm jq libcephfs-dev 2020-04-10T07:01:56.406 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs2 libjq1 libleveldb1v5 liblttng-ust0 liboath0 libonig4 librabbitmq4 2020-04-10T07:01:56.406 INFO:teuthology.orchestra.run.smithi204.stdout: libradosstriper1 librdkafka1 librgw2 libsnappy1v5 libxslt1.1 python-attr 2020-04-10T07:01:56.406 INFO:teuthology.orchestra.run.smithi204.stdout: python-funcsigs python-pastedeploy-tpl python-pluggy python-py python-pytest 2020-04-10T07:01:56.406 INFO:teuthology.orchestra.run.smithi204.stdout: python3-asn1crypto python3-attr python3-bcrypt python3-bs4 2020-04-10T07:01:56.406 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs 2020-04-10T07:01:56.406 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cffi-backend python3-cherrypy3 python3-cryptography python3-cycler 2020-04-10T07:01:56.406 INFO:teuthology.orchestra.run.smithi204.stdout: python3-dateutil python3-decorator python3-fasteners python3-html5lib 2020-04-10T07:01:56.407 INFO:teuthology.orchestra.run.smithi204.stdout: python3-httplib2 python3-joblib python3-json-pointer python3-jsonpatch 2020-04-10T07:01:56.407 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2020-04-10T07:01:56.407 INFO:teuthology.orchestra.run.smithi204.stdout: python3-markupsafe python3-matplotlib python3-monotonic python3-numpy 2020-04-10T07:01:56.407 INFO:teuthology.orchestra.run.smithi204.stdout: python3-oauth2client python3-olefile python3-openssl python3-paste 2020-04-10T07:01:56.407 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pastedeploy python3-pastescript python3-pecan python3-pil 2020-04-10T07:01:56.407 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pluggy python3-prettytable python3-py python3-pyasn1 2020-04-10T07:01:56.407 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1-modules python3-pyinotify python3-pyparsing python3-pytest 2020-04-10T07:01:56.408 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rados python3-rbd python3-repoze.lru python3-rgw python3-routes 2020-04-10T07:01:56.408 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rsa python3-scipy python3-setuptools python3-simplegeneric 2020-04-10T07:01:56.408 INFO:teuthology.orchestra.run.smithi204.stdout: python3-simplejson python3-singledispatch python3-sklearn 2020-04-10T07:01:56.408 INFO:teuthology.orchestra.run.smithi204.stdout: python3-sklearn-lib python3-sqlalchemy python3-tempita python3-tk python3-tz 2020-04-10T07:01:56.408 INFO:teuthology.orchestra.run.smithi204.stdout: python3-uritemplate python3-waitress python3-webencodings python3-webob 2020-04-10T07:01:56.408 INFO:teuthology.orchestra.run.smithi204.stdout: python3-websocket python3-webtest python3-werkzeug radosgw rbd-fuse 2020-04-10T07:01:56.408 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-mirror socat xmlstarlet 2020-04-10T07:01:56.409 INFO:teuthology.orchestra.run.smithi204.stdout:The following packages will be upgraded: 2020-04-10T07:01:56.409 INFO:teuthology.orchestra.run.smithi204.stdout: librados2 librbd1 2020-04-10T07:01:56.529 INFO:teuthology.orchestra.run.smithi101.stdout:2 upgraded, 110 newly installed, 0 to remove and 44 not upgraded. 2020-04-10T07:01:56.529 INFO:teuthology.orchestra.run.smithi101.stdout:Need to get 159 MB of archives. 2020-04-10T07:01:56.529 INFO:teuthology.orchestra.run.smithi101.stdout:After this operation, 737 MB of additional disk space will be used. 2020-04-10T07:01:56.530 INFO:teuthology.orchestra.run.smithi101.stdout:Get:1 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-uritemplate all 0.6-1ubuntu1 [5,126 B] 2020-04-10T07:01:56.566 INFO:teuthology.orchestra.run.smithi101.stdout:Get:2 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main amd64 librbd1 amd64 15.2.0-148-g39357d2-1bionic [1,615 kB] 2020-04-10T07:01:56.600 INFO:teuthology.orchestra.run.smithi204.stdout:2 upgraded, 110 newly installed, 0 to remove and 44 not upgraded. 2020-04-10T07:01:56.600 INFO:teuthology.orchestra.run.smithi204.stdout:Need to get 159 MB of archives. 2020-04-10T07:01:56.600 INFO:teuthology.orchestra.run.smithi204.stdout:After this operation, 737 MB of additional disk space will be used. 2020-04-10T07:01:56.600 INFO:teuthology.orchestra.run.smithi204.stdout:Get:1 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-uritemplate all 0.6-1ubuntu1 [5,126 B] 2020-04-10T07:01:56.625 INFO:teuthology.orchestra.run.smithi101.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liblttng-ust0 amd64 2.10.1-1 [154 kB] 2020-04-10T07:01:56.635 INFO:teuthology.orchestra.run.smithi204.stdout:Get:2 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main amd64 librbd1 amd64 15.2.0-148-g39357d2-1bionic [1,615 kB] 2020-04-10T07:01:56.700 INFO:teuthology.orchestra.run.smithi204.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liblttng-ust0 amd64 2.10.1-1 [154 kB] 2020-04-10T07:01:56.828 INFO:teuthology.orchestra.run.smithi101.stdout:Get:4 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main amd64 librados2 amd64 15.2.0-148-g39357d2-1bionic [3,047 kB] 2020-04-10T07:01:56.896 INFO:teuthology.orchestra.run.smithi101.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-prettytable all 0.7.2-3 [19.7 kB] 2020-04-10T07:01:57.134 INFO:teuthology.orchestra.run.smithi204.stdout:Get:4 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main amd64 librados2 amd64 15.2.0-148-g39357d2-1bionic [3,047 kB] 2020-04-10T07:01:57.134 INFO:teuthology.orchestra.run.smithi204.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-prettytable all 0.7.2-3 [19.7 kB] 2020-04-10T07:01:57.134 INFO:teuthology.orchestra.run.smithi204.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 librabbitmq4 amd64 0.8.0-1ubuntu0.18.04.2 [33.9 kB] 2020-04-10T07:01:57.134 INFO:teuthology.orchestra.run.smithi204.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic/universe amd64 librdkafka1 amd64 0.11.3-1build1 [293 kB] 2020-04-10T07:01:57.134 INFO:teuthology.orchestra.run.smithi204.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic/main amd64 libsnappy1v5 amd64 1.1.7-1 [16.0 kB] 2020-04-10T07:01:57.135 INFO:teuthology.orchestra.run.smithi204.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic/main amd64 libleveldb1v5 amd64 1.20-2 [136 kB] 2020-04-10T07:01:57.135 INFO:teuthology.orchestra.run.smithi101.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 librabbitmq4 amd64 0.8.0-1ubuntu0.18.04.2 [33.9 kB] 2020-04-10T07:01:57.135 INFO:teuthology.orchestra.run.smithi101.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic/universe amd64 librdkafka1 amd64 0.11.3-1build1 [293 kB] 2020-04-10T07:01:57.135 INFO:teuthology.orchestra.run.smithi101.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs2 amd64 15.2.0-148-g39357d2-1bionic [432 kB] 2020-04-10T07:01:57.135 INFO:teuthology.orchestra.run.smithi101.stdout:Get:9 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main amd64 python3-rados amd64 15.2.0-148-g39357d2-1bionic [332 kB] 2020-04-10T07:01:57.136 INFO:teuthology.orchestra.run.smithi101.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic/main amd64 libsnappy1v5 amd64 1.1.7-1 [16.0 kB] 2020-04-10T07:01:57.136 INFO:teuthology.orchestra.run.smithi101.stdout:Get:11 http://archive.ubuntu.com/ubuntu bionic/main amd64 libleveldb1v5 amd64 1.20-2 [136 kB] 2020-04-10T07:01:57.136 INFO:teuthology.orchestra.run.smithi101.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liboath0 amd64 2.6.1-1 [44.7 kB] 2020-04-10T07:01:57.136 INFO:teuthology.orchestra.run.smithi101.stdout:Get:13 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-dateutil all 2.6.1-1 [52.3 kB] 2020-04-10T07:01:57.136 INFO:teuthology.orchestra.run.smithi101.stdout:Get:14 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-asn1crypto all 0.24.0-1 [72.8 kB] 2020-04-10T07:01:57.136 INFO:teuthology.orchestra.run.smithi101.stdout:Get:15 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-cffi-backend amd64 1.11.5-1 [64.6 kB] 2020-04-10T07:01:57.136 INFO:teuthology.orchestra.run.smithi101.stdout:Get:16 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-cryptography amd64 2.1.4-1ubuntu1.3 [221 kB] 2020-04-10T07:01:57.137 INFO:teuthology.orchestra.run.smithi101.stdout:Get:17 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-argparse all 15.2.0-148-g39357d2-1bionic [20.6 kB] 2020-04-10T07:01:57.137 INFO:teuthology.orchestra.run.smithi101.stdout:Get:18 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main amd64 python3-cephfs amd64 15.2.0-148-g39357d2-1bionic [132 kB] 2020-04-10T07:01:57.137 INFO:teuthology.orchestra.run.smithi101.stdout:Get:19 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-common all 15.2.0-148-g39357d2-1bionic [22.8 kB] 2020-04-10T07:01:57.137 INFO:teuthology.orchestra.run.smithi101.stdout:Get:20 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main amd64 python3-rbd amd64 15.2.0-148-g39357d2-1bionic [281 kB] 2020-04-10T07:01:57.137 INFO:teuthology.orchestra.run.smithi101.stdout:Get:21 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main amd64 librgw2 amd64 15.2.0-148-g39357d2-1bionic [5,350 kB] 2020-04-10T07:01:57.137 INFO:teuthology.orchestra.run.smithi101.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-openssl all 17.5.0-1ubuntu1 [41.5 kB] 2020-04-10T07:01:57.137 INFO:teuthology.orchestra.run.smithi101.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-bcrypt amd64 3.1.4-2 [29.9 kB] 2020-04-10T07:01:57.138 INFO:teuthology.orchestra.run.smithi101.stdout:Get:24 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cherrypy3 all 8.9.1-2 [160 kB] 2020-04-10T07:01:57.145 INFO:teuthology.orchestra.run.smithi204.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liboath0 amd64 2.6.1-1 [44.7 kB] 2020-04-10T07:01:57.150 INFO:teuthology.orchestra.run.smithi101.stdout:Get:25 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jwt all 1.5.3+ds1-1 [15.9 kB] 2020-04-10T07:01:57.150 INFO:teuthology.orchestra.run.smithi101.stdout:Get:26 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-logutils all 0.3.3-5 [16.7 kB] 2020-04-10T07:01:57.153 INFO:teuthology.orchestra.run.smithi204.stdout:Get:11 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-dateutil all 2.6.1-1 [52.3 kB] 2020-04-10T07:01:57.162 INFO:teuthology.orchestra.run.smithi204.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-asn1crypto all 0.24.0-1 [72.8 kB] 2020-04-10T07:01:57.172 INFO:teuthology.orchestra.run.smithi204.stdout:Get:13 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-cffi-backend amd64 1.11.5-1 [64.6 kB] 2020-04-10T07:01:57.180 INFO:teuthology.orchestra.run.smithi204.stdout:Get:14 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-cryptography amd64 2.1.4-1ubuntu1.3 [221 kB] 2020-04-10T07:01:57.205 INFO:teuthology.orchestra.run.smithi204.stdout:Get:15 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-openssl all 17.5.0-1ubuntu1 [41.5 kB] 2020-04-10T07:01:57.209 INFO:teuthology.orchestra.run.smithi204.stdout:Get:16 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-bcrypt amd64 3.1.4-2 [29.9 kB] 2020-04-10T07:01:57.218 INFO:teuthology.orchestra.run.smithi101.stdout:Get:27 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-markupsafe amd64 1.0-1build1 [13.5 kB] 2020-04-10T07:01:57.219 INFO:teuthology.orchestra.run.smithi101.stdout:Get:28 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-mako all 1.0.7+ds1-1 [59.3 kB] 2020-04-10T07:01:57.220 INFO:teuthology.orchestra.run.smithi204.stdout:Get:17 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs2 amd64 15.2.0-148-g39357d2-1bionic [432 kB] 2020-04-10T07:01:57.221 INFO:teuthology.orchestra.run.smithi101.stdout:Get:29 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-simplegeneric all 0.8.1-1 [11.5 kB] 2020-04-10T07:01:57.224 INFO:teuthology.orchestra.run.smithi101.stdout:Get:30 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-singledispatch all 3.4.0.3-2 [7,022 B] 2020-04-10T07:01:57.224 INFO:teuthology.orchestra.run.smithi101.stdout:Get:31 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webob all 1:1.7.3-2fakesync1 [64.3 kB] 2020-04-10T07:01:57.225 INFO:teuthology.orchestra.run.smithi101.stdout:Get:32 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-bs4 all 4.6.0-1 [67.8 kB] 2020-04-10T07:01:57.229 INFO:teuthology.orchestra.run.smithi101.stdout:Get:33 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-waitress all 1.0.1-1 [53.4 kB] 2020-04-10T07:01:57.229 INFO:teuthology.orchestra.run.smithi204.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cherrypy3 all 8.9.1-2 [160 kB] 2020-04-10T07:01:57.242 INFO:teuthology.orchestra.run.smithi204.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jwt all 1.5.3+ds1-1 [15.9 kB] 2020-04-10T07:01:57.243 INFO:teuthology.orchestra.run.smithi204.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-logutils all 0.3.3-5 [16.7 kB] 2020-04-10T07:01:57.253 INFO:teuthology.orchestra.run.smithi204.stdout:Get:21 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main amd64 python3-rados amd64 15.2.0-148-g39357d2-1bionic [332 kB] 2020-04-10T07:01:57.264 INFO:teuthology.orchestra.run.smithi204.stdout:Get:22 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-argparse all 15.2.0-148-g39357d2-1bionic [20.6 kB] 2020-04-10T07:01:57.264 INFO:teuthology.orchestra.run.smithi204.stdout:Get:23 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main amd64 python3-cephfs amd64 15.2.0-148-g39357d2-1bionic [132 kB] 2020-04-10T07:01:57.285 INFO:teuthology.orchestra.run.smithi101.stdout:Get:34 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main amd64 python3-rgw amd64 15.2.0-148-g39357d2-1bionic [99.4 kB] 2020-04-10T07:01:57.288 INFO:teuthology.orchestra.run.smithi101.stdout:Get:35 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main amd64 libradosstriper1 amd64 15.2.0-148-g39357d2-1bionic [334 kB] 2020-04-10T07:01:57.299 INFO:teuthology.orchestra.run.smithi101.stdout:Get:36 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main amd64 ceph-common amd64 15.2.0-148-g39357d2-1bionic [18.6 MB] 2020-04-10T07:01:57.299 INFO:teuthology.orchestra.run.smithi101.stdout:Get:37 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tempita all 0.5.2-2 [13.9 kB] 2020-04-10T07:01:57.300 INFO:teuthology.orchestra.run.smithi101.stdout:Get:38 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-paste all 2.0.3+dfsg-4ubuntu1 [456 kB] 2020-04-10T07:01:57.303 INFO:teuthology.orchestra.run.smithi204.stdout:Get:24 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-common all 15.2.0-148-g39357d2-1bionic [22.8 kB] 2020-04-10T07:01:57.310 INFO:teuthology.orchestra.run.smithi204.stdout:Get:25 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-markupsafe amd64 1.0-1build1 [13.5 kB] 2020-04-10T07:01:57.311 INFO:teuthology.orchestra.run.smithi204.stdout:Get:26 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-mako all 1.0.7+ds1-1 [59.3 kB] 2020-04-10T07:01:57.314 INFO:teuthology.orchestra.run.smithi204.stdout:Get:27 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-simplegeneric all 0.8.1-1 [11.5 kB] 2020-04-10T07:01:57.314 INFO:teuthology.orchestra.run.smithi204.stdout:Get:28 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-singledispatch all 3.4.0.3-2 [7,022 B] 2020-04-10T07:01:57.315 INFO:teuthology.orchestra.run.smithi204.stdout:Get:29 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webob all 1:1.7.3-2fakesync1 [64.3 kB] 2020-04-10T07:01:57.317 INFO:teuthology.orchestra.run.smithi204.stdout:Get:30 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-bs4 all 4.6.0-1 [67.8 kB] 2020-04-10T07:01:57.320 INFO:teuthology.orchestra.run.smithi101.stdout:Get:39 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-pastedeploy-tpl all 1.5.2-4 [4,796 B] 2020-04-10T07:01:57.321 INFO:teuthology.orchestra.run.smithi204.stdout:Get:31 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-waitress all 1.0.1-1 [53.4 kB] 2020-04-10T07:01:57.328 INFO:teuthology.orchestra.run.smithi204.stdout:Get:32 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main amd64 python3-rbd amd64 15.2.0-148-g39357d2-1bionic [281 kB] 2020-04-10T07:01:57.332 INFO:teuthology.orchestra.run.smithi204.stdout:Get:33 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main amd64 librgw2 amd64 15.2.0-148-g39357d2-1bionic [5,350 kB] 2020-04-10T07:01:57.380 INFO:teuthology.orchestra.run.smithi101.stdout:Get:40 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastedeploy all 1.5.2-4 [13.4 kB] 2020-04-10T07:01:57.381 INFO:teuthology.orchestra.run.smithi101.stdout:Get:41 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webtest all 2.0.28-1ubuntu1 [27.9 kB] 2020-04-10T07:01:57.382 INFO:teuthology.orchestra.run.smithi101.stdout:Get:42 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pecan all 1.2.1-2 [86.1 kB] 2020-04-10T07:01:57.386 INFO:teuthology.orchestra.run.smithi101.stdout:Get:43 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-werkzeug all 0.14.1+dfsg1-1 [174 kB] 2020-04-10T07:01:57.391 INFO:teuthology.orchestra.run.smithi204.stdout:Get:34 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tempita all 0.5.2-2 [13.9 kB] 2020-04-10T07:01:57.392 INFO:teuthology.orchestra.run.smithi204.stdout:Get:35 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-paste all 2.0.3+dfsg-4ubuntu1 [456 kB] 2020-04-10T07:01:57.393 INFO:teuthology.orchestra.run.smithi101.stdout:Get:44 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-repoze.lru all 0.7-1 [11.7 kB] 2020-04-10T07:01:57.393 INFO:teuthology.orchestra.run.smithi101.stdout:Get:45 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-routes all 2.4.1-1 [28.4 kB] 2020-04-10T07:01:57.395 INFO:teuthology.orchestra.run.smithi101.stdout:Get:46 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-numpy amd64 1:1.13.3-2ubuntu1 [1,943 kB] 2020-04-10T07:01:57.412 INFO:teuthology.orchestra.run.smithi204.stdout:Get:36 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-pastedeploy-tpl all 1.5.2-4 [4,796 B] 2020-04-10T07:01:57.472 INFO:teuthology.orchestra.run.smithi204.stdout:Get:37 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastedeploy all 1.5.2-4 [13.4 kB] 2020-04-10T07:01:57.473 INFO:teuthology.orchestra.run.smithi204.stdout:Get:38 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webtest all 2.0.28-1ubuntu1 [27.9 kB] 2020-04-10T07:01:57.475 INFO:teuthology.orchestra.run.smithi204.stdout:Get:39 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pecan all 1.2.1-2 [86.1 kB] 2020-04-10T07:01:57.478 INFO:teuthology.orchestra.run.smithi204.stdout:Get:40 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-werkzeug all 0.14.1+dfsg1-1 [174 kB] 2020-04-10T07:01:57.480 INFO:teuthology.orchestra.run.smithi101.stdout:Get:47 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-decorator all 4.1.2-1 [9,364 B] 2020-04-10T07:01:57.480 INFO:teuthology.orchestra.run.smithi101.stdout:Get:48 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-scipy amd64 0.19.1-2ubuntu1 [9,619 kB] 2020-04-10T07:01:57.486 INFO:teuthology.orchestra.run.smithi204.stdout:Get:41 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-repoze.lru all 0.7-1 [11.7 kB] 2020-04-10T07:01:57.486 INFO:teuthology.orchestra.run.smithi204.stdout:Get:42 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-routes all 2.4.1-1 [28.4 kB] 2020-04-10T07:01:57.488 INFO:teuthology.orchestra.run.smithi204.stdout:Get:43 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-numpy amd64 1:1.13.3-2ubuntu1 [1,943 kB] 2020-04-10T07:01:57.574 INFO:teuthology.orchestra.run.smithi204.stdout:Get:44 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-decorator all 4.1.2-1 [9,364 B] 2020-04-10T07:01:57.574 INFO:teuthology.orchestra.run.smithi204.stdout:Get:45 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-scipy amd64 0.19.1-2ubuntu1 [9,619 kB] 2020-04-10T07:01:57.767 INFO:teuthology.orchestra.run.smithi101.stdout:Get:49 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn-lib amd64 0.19.1-3 [1,429 kB] 2020-04-10T07:01:57.783 INFO:teuthology.orchestra.run.smithi204.stdout:Get:46 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main amd64 python3-rgw amd64 15.2.0-148-g39357d2-1bionic [99.4 kB] 2020-04-10T07:01:57.785 INFO:teuthology.orchestra.run.smithi204.stdout:Get:47 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main amd64 libradosstriper1 amd64 15.2.0-148-g39357d2-1bionic [334 kB] 2020-04-10T07:01:57.805 INFO:teuthology.orchestra.run.smithi101.stdout:Get:50 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-joblib all 0.11-1 [109 kB] 2020-04-10T07:01:57.806 INFO:teuthology.orchestra.run.smithi101.stdout:Get:51 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn all 0.19.1-3 [1,373 kB] 2020-04-10T07:01:57.842 INFO:teuthology.orchestra.run.smithi101.stdout:Get:52 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-monotonic all 1.1-2 [5,332 B] 2020-04-10T07:01:57.842 INFO:teuthology.orchestra.run.smithi101.stdout:Get:53 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-fasteners all 0.12.0-3 [13.5 kB] 2020-04-10T07:01:57.843 INFO:teuthology.orchestra.run.smithi101.stdout:Get:54 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-httplib2 all 0.9.2+dfsg-1ubuntu0.1 [28.6 kB] 2020-04-10T07:01:57.843 INFO:teuthology.orchestra.run.smithi101.stdout:Get:55 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1 all 0.4.2-3 [46.8 kB] 2020-04-10T07:01:57.843 INFO:teuthology.orchestra.run.smithi101.stdout:Get:56 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1-modules all 0.2.1-0.2 [32.9 kB] 2020-04-10T07:01:57.844 INFO:teuthology.orchestra.run.smithi101.stdout:Get:57 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-rsa all 3.4.2-1 [29.0 kB] 2020-04-10T07:01:57.849 INFO:teuthology.orchestra.run.smithi101.stdout:Get:58 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-sqlalchemy all 1.1.11+ds1-1ubuntu1 [680 kB] 2020-04-10T07:01:57.853 INFO:teuthology.orchestra.run.smithi204.stdout:Get:48 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main amd64 ceph-common amd64 15.2.0-148-g39357d2-1bionic [18.6 MB] 2020-04-10T07:01:57.857 INFO:teuthology.orchestra.run.smithi204.stdout:Get:49 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn-lib amd64 0.19.1-3 [1,429 kB] 2020-04-10T07:01:57.887 INFO:teuthology.orchestra.run.smithi204.stdout:Get:50 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-joblib all 0.11-1 [109 kB] 2020-04-10T07:01:57.888 INFO:teuthology.orchestra.run.smithi204.stdout:Get:51 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn all 0.19.1-3 [1,373 kB] 2020-04-10T07:01:57.925 INFO:teuthology.orchestra.run.smithi101.stdout:Get:59 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-oauth2client all 4.1.2-3 [61.5 kB] 2020-04-10T07:01:57.928 INFO:teuthology.orchestra.run.smithi101.stdout:Get:60 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-setuptools all 39.0.1-2 [248 kB] 2020-04-10T07:01:57.934 INFO:teuthology.orchestra.run.smithi204.stdout:Get:52 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-monotonic all 1.1-2 [5,332 B] 2020-04-10T07:01:57.935 INFO:teuthology.orchestra.run.smithi204.stdout:Get:53 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-fasteners all 0.12.0-3 [13.5 kB] 2020-04-10T07:01:57.935 INFO:teuthology.orchestra.run.smithi101.stdout:Get:61 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-websocket all 0.44.0-0ubuntu2 [30.7 kB] 2020-04-10T07:01:57.936 INFO:teuthology.orchestra.run.smithi204.stdout:Get:54 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-httplib2 all 0.9.2+dfsg-1ubuntu0.1 [28.6 kB] 2020-04-10T07:01:57.936 INFO:teuthology.orchestra.run.smithi101.stdout:Get:62 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-kubernetes all 2.0.0-2ubuntu1 [181 kB] 2020-04-10T07:01:57.937 INFO:teuthology.orchestra.run.smithi204.stdout:Get:55 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1 all 0.4.2-3 [46.8 kB] 2020-04-10T07:01:57.938 INFO:teuthology.orchestra.run.smithi204.stdout:Get:56 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1-modules all 0.2.1-0.2 [32.9 kB] 2020-04-10T07:01:57.939 INFO:teuthology.orchestra.run.smithi101.stdout:Get:63 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-json-pointer all 1.10-1 [7,478 B] 2020-04-10T07:01:57.939 INFO:teuthology.orchestra.run.smithi101.stdout:Get:64 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jsonpatch all 1.19+really1.16-1fakesync1 [13.4 kB] 2020-04-10T07:01:57.939 INFO:teuthology.orchestra.run.smithi101.stdout:Get:65 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libonig4 amd64 6.7.0-1 [119 kB] 2020-04-10T07:01:57.940 INFO:teuthology.orchestra.run.smithi101.stdout:Get:66 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libjq1 amd64 1.5+dfsg-2 [111 kB] 2020-04-10T07:01:57.941 INFO:teuthology.orchestra.run.smithi204.stdout:Get:57 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-rsa all 3.4.2-1 [29.0 kB] 2020-04-10T07:01:57.941 INFO:teuthology.orchestra.run.smithi204.stdout:Get:58 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-sqlalchemy all 1.1.11+ds1-1ubuntu1 [680 kB] 2020-04-10T07:01:57.941 INFO:teuthology.orchestra.run.smithi101.stdout:Get:67 http://archive.ubuntu.com/ubuntu bionic/universe amd64 jq amd64 1.5+dfsg-2 [45.6 kB] 2020-04-10T07:01:58.012 INFO:teuthology.orchestra.run.smithi101.stdout:Get:68 http://archive.ubuntu.com/ubuntu bionic/main amd64 socat amd64 1.7.3.2-2ubuntu2 [342 kB] 2020-04-10T07:01:58.020 INFO:teuthology.orchestra.run.smithi204.stdout:Get:59 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-oauth2client all 4.1.2-3 [61.5 kB] 2020-04-10T07:01:58.025 INFO:teuthology.orchestra.run.smithi204.stdout:Get:60 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-setuptools all 39.0.1-2 [248 kB] 2020-04-10T07:01:58.029 INFO:teuthology.orchestra.run.smithi204.stdout:Get:61 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-websocket all 0.44.0-0ubuntu2 [30.7 kB] 2020-04-10T07:01:58.031 INFO:teuthology.orchestra.run.smithi204.stdout:Get:62 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-kubernetes all 2.0.0-2ubuntu1 [181 kB] 2020-04-10T07:01:58.035 INFO:teuthology.orchestra.run.smithi204.stdout:Get:63 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-json-pointer all 1.10-1 [7,478 B] 2020-04-10T07:01:58.036 INFO:teuthology.orchestra.run.smithi204.stdout:Get:64 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jsonpatch all 1.19+really1.16-1fakesync1 [13.4 kB] 2020-04-10T07:01:58.092 INFO:teuthology.orchestra.run.smithi101.stdout:Get:69 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libxslt1.1 amd64 1.1.29-5ubuntu0.2 [150 kB] 2020-04-10T07:01:58.097 INFO:teuthology.orchestra.run.smithi101.stdout:Get:70 http://archive.ubuntu.com/ubuntu bionic/universe amd64 xmlstarlet amd64 1.6.1-2 [382 kB] 2020-04-10T07:01:58.101 INFO:teuthology.orchestra.run.smithi204.stdout:Get:65 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libonig4 amd64 6.7.0-1 [119 kB] 2020-04-10T07:01:58.102 INFO:teuthology.orchestra.run.smithi101.stdout:Get:71 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-attr all 17.4.0-2 [23.7 kB] 2020-04-10T07:01:58.103 INFO:teuthology.orchestra.run.smithi101.stdout:Get:72 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-funcsigs all 1.0.2-4 [13.5 kB] 2020-04-10T07:01:58.103 INFO:teuthology.orchestra.run.smithi101.stdout:Get:73 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pluggy all 0.6.0-1 [12.2 kB] 2020-04-10T07:01:58.104 INFO:teuthology.orchestra.run.smithi101.stdout:Get:74 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-py all 1.5.2-1 [65.1 kB] 2020-04-10T07:01:58.104 INFO:teuthology.orchestra.run.smithi204.stdout:Get:66 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libjq1 amd64 1.5+dfsg-2 [111 kB] 2020-04-10T07:01:58.105 INFO:teuthology.orchestra.run.smithi101.stdout:Get:75 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pytest all 3.3.2-2 [142 kB] 2020-04-10T07:01:58.107 INFO:teuthology.orchestra.run.smithi204.stdout:Get:67 http://archive.ubuntu.com/ubuntu bionic/universe amd64 jq amd64 1.5+dfsg-2 [45.6 kB] 2020-04-10T07:01:58.107 INFO:teuthology.orchestra.run.smithi101.stdout:Get:76 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-attr all 17.4.0-2 [23.8 kB] 2020-04-10T07:01:58.108 INFO:teuthology.orchestra.run.smithi101.stdout:Get:77 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cycler all 0.10.0-1 [7,622 B] 2020-04-10T07:01:58.108 INFO:teuthology.orchestra.run.smithi204.stdout:Get:68 http://archive.ubuntu.com/ubuntu bionic/main amd64 socat amd64 1.7.3.2-2ubuntu2 [342 kB] 2020-04-10T07:01:58.173 INFO:teuthology.orchestra.run.smithi101.stdout:Get:78 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webencodings all 0.5-2 [10.4 kB] 2020-04-10T07:01:58.182 INFO:teuthology.orchestra.run.smithi204.stdout:Get:69 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libxslt1.1 amd64 1.1.29-5ubuntu0.2 [150 kB] 2020-04-10T07:01:58.184 INFO:teuthology.orchestra.run.smithi204.stdout:Get:70 http://archive.ubuntu.com/ubuntu bionic/universe amd64 xmlstarlet amd64 1.6.1-2 [382 kB] 2020-04-10T07:01:58.193 INFO:teuthology.orchestra.run.smithi204.stdout:Get:71 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-attr all 17.4.0-2 [23.7 kB] 2020-04-10T07:01:58.194 INFO:teuthology.orchestra.run.smithi204.stdout:Get:72 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-funcsigs all 1.0.2-4 [13.5 kB] 2020-04-10T07:01:58.194 INFO:teuthology.orchestra.run.smithi204.stdout:Get:73 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pluggy all 0.6.0-1 [12.2 kB] 2020-04-10T07:01:58.194 INFO:teuthology.orchestra.run.smithi204.stdout:Get:74 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-py all 1.5.2-1 [65.1 kB] 2020-04-10T07:01:58.254 INFO:teuthology.orchestra.run.smithi101.stdout:Get:79 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-html5lib all 0.999999999-1 [81.9 kB] 2020-04-10T07:01:58.258 INFO:teuthology.orchestra.run.smithi101.stdout:Get:80 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-lxml amd64 4.2.1-1ubuntu0.1 [1,091 kB] 2020-04-10T07:01:58.262 INFO:teuthology.orchestra.run.smithi204.stdout:Get:75 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pytest all 3.3.2-2 [142 kB] 2020-04-10T07:01:58.265 INFO:teuthology.orchestra.run.smithi204.stdout:Get:76 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-attr all 17.4.0-2 [23.8 kB] 2020-04-10T07:01:58.266 INFO:teuthology.orchestra.run.smithi204.stdout:Get:77 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cycler all 0.10.0-1 [7,622 B] 2020-04-10T07:01:58.266 INFO:teuthology.orchestra.run.smithi204.stdout:Get:78 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webencodings all 0.5-2 [10.4 kB] 2020-04-10T07:01:58.273 INFO:teuthology.orchestra.run.smithi101.stdout:Get:81 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main amd64 ceph-base amd64 15.2.0-148-g39357d2-1bionic [5,129 kB] 2020-04-10T07:01:58.277 INFO:teuthology.orchestra.run.smithi101.stdout:Get:82 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyparsing all 2.2.0+dfsg1-2 [52.2 kB] 2020-04-10T07:01:58.278 INFO:teuthology.orchestra.run.smithi101.stdout:Get:83 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tz all 2018.3-2 [25.1 kB] 2020-04-10T07:01:58.279 INFO:teuthology.orchestra.run.smithi101.stdout:Get:84 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-matplotlib amd64 2.1.1-2ubuntu3 [3,907 kB] 2020-04-10T07:01:58.343 INFO:teuthology.orchestra.run.smithi204.stdout:Get:79 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-html5lib all 0.999999999-1 [81.9 kB] 2020-04-10T07:01:58.345 INFO:teuthology.orchestra.run.smithi204.stdout:Get:80 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-lxml amd64 4.2.1-1ubuntu0.1 [1,091 kB] 2020-04-10T07:01:58.361 INFO:teuthology.orchestra.run.smithi204.stdout:Get:81 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyparsing all 2.2.0+dfsg1-2 [52.2 kB] 2020-04-10T07:01:58.362 INFO:teuthology.orchestra.run.smithi204.stdout:Get:82 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tz all 2018.3-2 [25.1 kB] 2020-04-10T07:01:58.362 INFO:teuthology.orchestra.run.smithi204.stdout:Get:83 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-matplotlib amd64 2.1.1-2ubuntu3 [3,907 kB] 2020-04-10T07:01:58.420 INFO:teuthology.orchestra.run.smithi101.stdout:Get:85 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-olefile all 0.45.1-1 [33.3 kB] 2020-04-10T07:01:58.421 INFO:teuthology.orchestra.run.smithi101.stdout:Get:86 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastescript all 2.0.2-2 [54.6 kB] 2020-04-10T07:01:58.422 INFO:teuthology.orchestra.run.smithi101.stdout:Get:87 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-pil amd64 5.1.0-1ubuntu0.2 [329 kB] 2020-04-10T07:01:58.429 INFO:teuthology.orchestra.run.smithi101.stdout:Get:88 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pluggy all 0.6.0-1 [12.3 kB] 2020-04-10T07:01:58.430 INFO:teuthology.orchestra.run.smithi101.stdout:Get:89 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-py all 1.5.2-1 [65.2 kB] 2020-04-10T07:01:58.430 INFO:teuthology.orchestra.run.smithi101.stdout:Get:90 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyinotify all 0.9.6-1 [24.7 kB] 2020-04-10T07:01:58.431 INFO:teuthology.orchestra.run.smithi101.stdout:Get:91 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pytest all 3.3.2-2 [142 kB] 2020-04-10T07:01:58.432 INFO:teuthology.orchestra.run.smithi101.stdout:Get:92 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-simplejson amd64 3.13.2-1 [49.6 kB] 2020-04-10T07:01:58.436 INFO:teuthology.orchestra.run.smithi101.stdout:Get:93 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-tk amd64 3.6.9-1~18.04 [119 kB] 2020-04-10T07:01:58.451 INFO:teuthology.orchestra.run.smithi204.stdout:Get:84 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-olefile all 0.45.1-1 [33.3 kB] 2020-04-10T07:01:58.452 INFO:teuthology.orchestra.run.smithi204.stdout:Get:85 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastescript all 2.0.2-2 [54.6 kB] 2020-04-10T07:01:58.452 INFO:teuthology.orchestra.run.smithi204.stdout:Get:86 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-pil amd64 5.1.0-1ubuntu0.2 [329 kB] 2020-04-10T07:01:58.506 INFO:teuthology.orchestra.run.smithi204.stdout:Get:87 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pluggy all 0.6.0-1 [12.3 kB] 2020-04-10T07:01:58.507 INFO:teuthology.orchestra.run.smithi204.stdout:Get:88 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-py all 1.5.2-1 [65.2 kB] 2020-04-10T07:01:58.509 INFO:teuthology.orchestra.run.smithi204.stdout:Get:89 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyinotify all 0.9.6-1 [24.7 kB] 2020-04-10T07:01:58.510 INFO:teuthology.orchestra.run.smithi204.stdout:Get:90 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pytest all 3.3.2-2 [142 kB] 2020-04-10T07:01:58.514 INFO:teuthology.orchestra.run.smithi204.stdout:Get:91 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-simplejson amd64 3.13.2-1 [49.6 kB] 2020-04-10T07:01:58.515 INFO:teuthology.orchestra.run.smithi204.stdout:Get:92 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-tk amd64 3.6.9-1~18.04 [119 kB] 2020-04-10T07:01:58.547 INFO:teuthology.orchestra.run.smithi101.stdout:Get:94 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-modules-core all 15.2.0-148-g39357d2-1bionic [136 kB] 2020-04-10T07:01:58.656 INFO:teuthology.orchestra.run.smithi101.stdout:Get:95 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr amd64 15.2.0-148-g39357d2-1bionic [1,278 kB] 2020-04-10T07:01:58.670 INFO:teuthology.orchestra.run.smithi101.stdout:Get:96 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mon amd64 15.2.0-148-g39357d2-1bionic [5,940 kB] 2020-04-10T07:01:58.933 INFO:teuthology.orchestra.run.smithi101.stdout:Get:97 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main amd64 ceph-osd amd64 15.2.0-148-g39357d2-1bionic [22.7 MB] 2020-04-10T07:01:59.466 INFO:teuthology.orchestra.run.smithi204.stdout:Get:93 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main amd64 ceph-base amd64 15.2.0-148-g39357d2-1bionic [5,129 kB] 2020-04-10T07:01:59.904 INFO:teuthology.orchestra.run.smithi204.stdout:Get:94 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-modules-core all 15.2.0-148-g39357d2-1bionic [136 kB] 2020-04-10T07:01:59.935 INFO:teuthology.orchestra.run.smithi204.stdout:Get:95 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr amd64 15.2.0-148-g39357d2-1bionic [1,278 kB] 2020-04-10T07:02:00.042 INFO:teuthology.orchestra.run.smithi204.stdout:Get:96 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mon amd64 15.2.0-148-g39357d2-1bionic [5,940 kB] 2020-04-10T07:02:00.133 INFO:teuthology.orchestra.run.smithi101.stdout:Get:98 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main amd64 ceph amd64 15.2.0-148-g39357d2-1bionic [3,804 B] 2020-04-10T07:02:00.133 INFO:teuthology.orchestra.run.smithi101.stdout:Get:99 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main amd64 ceph-fuse amd64 15.2.0-148-g39357d2-1bionic [513 kB] 2020-04-10T07:02:00.147 INFO:teuthology.orchestra.run.smithi101.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mds amd64 15.2.0-148-g39357d2-1bionic [1,837 kB] 2020-04-10T07:02:00.237 INFO:teuthology.orchestra.run.smithi101.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main amd64 cephadm amd64 15.2.0-148-g39357d2-1bionic [34.6 kB] 2020-04-10T07:02:00.239 INFO:teuthology.orchestra.run.smithi101.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-cephadm all 15.2.0-148-g39357d2-1bionic [34.4 kB] 2020-04-10T07:02:00.240 INFO:teuthology.orchestra.run.smithi101.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-dashboard all 15.2.0-148-g39357d2-1bionic [2,915 kB] 2020-04-10T07:02:00.392 INFO:teuthology.orchestra.run.smithi101.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-diskprediction-cloud all 15.2.0-148-g39357d2-1bionic [34.2 kB] 2020-04-10T07:02:00.393 INFO:teuthology.orchestra.run.smithi101.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-diskprediction-local all 15.2.0-148-g39357d2-1bionic [7,864 kB] 2020-04-10T07:02:00.564 INFO:teuthology.orchestra.run.smithi204.stdout:Get:97 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main amd64 ceph-osd amd64 15.2.0-148-g39357d2-1bionic [22.7 MB] 2020-04-10T07:02:00.820 INFO:teuthology.orchestra.run.smithi101.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-k8sevents all 15.2.0-148-g39357d2-1bionic [3,920 B] 2020-04-10T07:02:00.820 INFO:teuthology.orchestra.run.smithi101.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-rook all 15.2.0-148-g39357d2-1bionic [22.3 kB] 2020-04-10T07:02:00.821 INFO:teuthology.orchestra.run.smithi101.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main amd64 ceph-test amd64 15.2.0-148-g39357d2-1bionic [43.3 MB] 2020-04-10T07:02:02.530 INFO:teuthology.orchestra.run.smithi204.stdout:Get:98 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main amd64 ceph amd64 15.2.0-148-g39357d2-1bionic [3,804 B] 2020-04-10T07:02:02.530 INFO:teuthology.orchestra.run.smithi204.stdout:Get:99 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main amd64 ceph-fuse amd64 15.2.0-148-g39357d2-1bionic [513 kB] 2020-04-10T07:02:02.595 INFO:teuthology.orchestra.run.smithi204.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mds amd64 15.2.0-148-g39357d2-1bionic [1,837 kB] 2020-04-10T07:02:02.756 INFO:teuthology.orchestra.run.smithi204.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main amd64 cephadm amd64 15.2.0-148-g39357d2-1bionic [34.6 kB] 2020-04-10T07:02:02.757 INFO:teuthology.orchestra.run.smithi204.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-cephadm all 15.2.0-148-g39357d2-1bionic [34.4 kB] 2020-04-10T07:02:02.758 INFO:teuthology.orchestra.run.smithi204.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-dashboard all 15.2.0-148-g39357d2-1bionic [2,915 kB] 2020-04-10T07:02:03.005 INFO:teuthology.orchestra.run.smithi204.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-diskprediction-cloud all 15.2.0-148-g39357d2-1bionic [34.2 kB] 2020-04-10T07:02:03.006 INFO:teuthology.orchestra.run.smithi204.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-diskprediction-local all 15.2.0-148-g39357d2-1bionic [7,864 kB] 2020-04-10T07:02:03.110 INFO:teuthology.orchestra.run.smithi101.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs-dev amd64 15.2.0-148-g39357d2-1bionic [17.4 kB] 2020-04-10T07:02:03.110 INFO:teuthology.orchestra.run.smithi101.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main amd64 radosgw amd64 15.2.0-148-g39357d2-1bionic [8,486 kB] 2020-04-10T07:02:03.536 INFO:teuthology.orchestra.run.smithi101.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main amd64 rbd-fuse amd64 15.2.0-148-g39357d2-1bionic [62.2 kB] 2020-04-10T07:02:03.537 INFO:teuthology.orchestra.run.smithi101.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main amd64 rbd-mirror amd64 15.2.0-148-g39357d2-1bionic [2,397 kB] 2020-04-10T07:02:03.691 INFO:teuthology.orchestra.run.smithi204.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-k8sevents all 15.2.0-148-g39357d2-1bionic [3,920 B] 2020-04-10T07:02:03.691 INFO:teuthology.orchestra.run.smithi204.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-rook all 15.2.0-148-g39357d2-1bionic [22.3 kB] 2020-04-10T07:02:03.691 INFO:teuthology.orchestra.run.smithi204.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main amd64 ceph-test amd64 15.2.0-148-g39357d2-1bionic [43.3 MB] 2020-04-10T07:02:04.370 INFO:teuthology.orchestra.run.smithi101.stdout:Fetched 159 MB in 7s (21.7 MB/s) 2020-04-10T07:02:04.519 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-uritemplate. 2020-04-10T07:02:04.568 INFO:teuthology.orchestra.run.smithi101.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 137475 files and directories currently installed.) 2020-04-10T07:02:04.572 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../000-python3-uritemplate_0.6-1ubuntu1_all.deb ... 2020-04-10T07:02:04.624 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-uritemplate (0.6-1ubuntu1) ... 2020-04-10T07:02:04.920 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2020-04-10T07:02:04.935 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../001-liblttng-ust0_2.10.1-1_amd64.deb ... 2020-04-10T07:02:05.021 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking liblttng-ust0:amd64 (2.10.1-1) ... 2020-04-10T07:02:05.356 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../002-librbd1_15.2.0-148-g39357d2-1bionic_amd64.deb ... 2020-04-10T07:02:05.484 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking librbd1 (15.2.0-148-g39357d2-1bionic) over (12.2.12-0ubuntu0.18.04.5) ... 2020-04-10T07:02:06.013 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../003-librados2_15.2.0-148-g39357d2-1bionic_amd64.deb ... 2020-04-10T07:02:06.095 INFO:teuthology.orchestra.run.smithi204.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs-dev amd64 15.2.0-148-g39357d2-1bionic [17.4 kB] 2020-04-10T07:02:06.095 INFO:teuthology.orchestra.run.smithi204.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main amd64 radosgw amd64 15.2.0-148-g39357d2-1bionic [8,486 kB] 2020-04-10T07:02:06.141 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking librados2 (15.2.0-148-g39357d2-1bionic) over (12.2.12-0ubuntu0.18.04.5) ... 2020-04-10T07:02:06.499 INFO:teuthology.orchestra.run.smithi204.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main amd64 rbd-fuse amd64 15.2.0-148-g39357d2-1bionic [62.2 kB] 2020-04-10T07:02:06.500 INFO:teuthology.orchestra.run.smithi204.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/octopus/39357d2a782a02d1ea09c76cbe7bc6f222a1127a/ubuntu/bionic/flavors/default bionic/main amd64 rbd-mirror amd64 15.2.0-148-g39357d2-1bionic [2,397 kB] 2020-04-10T07:02:06.803 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package libcephfs2. 2020-04-10T07:02:06.818 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../004-libcephfs2_15.2.0-148-g39357d2-1bionic_amd64.deb ... 2020-04-10T07:02:06.852 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking libcephfs2 (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:02:07.105 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-rados. 2020-04-10T07:02:07.119 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../005-python3-rados_15.2.0-148-g39357d2-1bionic_amd64.deb ... 2020-04-10T07:02:07.155 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-rados (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:02:07.312 INFO:teuthology.orchestra.run.smithi204.stdout:Fetched 159 MB in 10s (15.6 MB/s) 2020-04-10T07:02:07.463 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-uritemplate. 2020-04-10T07:02:07.514 INFO:teuthology.orchestra.run.smithi204.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 137475 files and directories currently installed.) 2020-04-10T07:02:07.518 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../000-python3-uritemplate_0.6-1ubuntu1_all.deb ... 2020-04-10T07:02:07.532 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-ceph-argparse. 2020-04-10T07:02:07.548 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../006-python3-ceph-argparse_15.2.0-148-g39357d2-1bionic_all.deb ... 2020-04-10T07:02:07.562 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-uritemplate (0.6-1ubuntu1) ... 2020-04-10T07:02:07.591 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-ceph-argparse (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:02:07.857 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2020-04-10T07:02:07.870 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../001-liblttng-ust0_2.10.1-1_amd64.deb ... 2020-04-10T07:02:07.910 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-cephfs. 2020-04-10T07:02:07.926 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../007-python3-cephfs_15.2.0-148-g39357d2-1bionic_amd64.deb ... 2020-04-10T07:02:07.940 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking liblttng-ust0:amd64 (2.10.1-1) ... 2020-04-10T07:02:07.977 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-cephfs (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:02:08.288 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-ceph-common. 2020-04-10T07:02:08.301 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../008-python3-ceph-common_15.2.0-148-g39357d2-1bionic_all.deb ... 2020-04-10T07:02:08.338 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-ceph-common (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:02:08.482 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../002-librbd1_15.2.0-148-g39357d2-1bionic_amd64.deb ... 2020-04-10T07:02:08.637 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking librbd1 (15.2.0-148-g39357d2-1bionic) over (12.2.12-0ubuntu0.18.04.5) ... 2020-04-10T07:02:08.716 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-prettytable. 2020-04-10T07:02:08.730 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../009-python3-prettytable_0.7.2-3_all.deb ... 2020-04-10T07:02:08.774 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-prettytable (0.7.2-3) ... 2020-04-10T07:02:09.060 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-rbd. 2020-04-10T07:02:09.073 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../010-python3-rbd_15.2.0-148-g39357d2-1bionic_amd64.deb ... 2020-04-10T07:02:09.110 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-rbd (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:02:09.142 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../003-librados2_15.2.0-148-g39357d2-1bionic_amd64.deb ... 2020-04-10T07:02:09.243 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking librados2 (15.2.0-148-g39357d2-1bionic) over (12.2.12-0ubuntu0.18.04.5) ... 2020-04-10T07:02:09.529 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package librabbitmq4:amd64. 2020-04-10T07:02:09.544 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../011-librabbitmq4_0.8.0-1ubuntu0.18.04.2_amd64.deb ... 2020-04-10T07:02:09.579 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2020-04-10T07:02:09.874 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package librdkafka1:amd64. 2020-04-10T07:02:09.880 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libcephfs2. 2020-04-10T07:02:09.888 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../012-librdkafka1_0.11.3-1build1_amd64.deb ... 2020-04-10T07:02:09.894 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../004-libcephfs2_15.2.0-148-g39357d2-1bionic_amd64.deb ... 2020-04-10T07:02:09.932 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking librdkafka1:amd64 (0.11.3-1build1) ... 2020-04-10T07:02:09.938 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libcephfs2 (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:02:10.251 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package librgw2. 2020-04-10T07:02:10.266 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-rados. 2020-04-10T07:02:10.267 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../013-librgw2_15.2.0-148-g39357d2-1bionic_amd64.deb ... 2020-04-10T07:02:10.279 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../005-python3-rados_15.2.0-148-g39357d2-1bionic_amd64.deb ... 2020-04-10T07:02:10.309 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking librgw2 (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:02:10.316 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-rados (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:02:10.647 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-ceph-argparse. 2020-04-10T07:02:10.659 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../006-python3-ceph-argparse_15.2.0-148-g39357d2-1bionic_all.deb ... 2020-04-10T07:02:10.702 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-ceph-argparse (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:02:11.012 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-rgw. 2020-04-10T07:02:11.025 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../014-python3-rgw_15.2.0-148-g39357d2-1bionic_amd64.deb ... 2020-04-10T07:02:11.030 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-cephfs. 2020-04-10T07:02:11.043 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../007-python3-cephfs_15.2.0-148-g39357d2-1bionic_amd64.deb ... 2020-04-10T07:02:11.062 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-rgw (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:02:11.080 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-cephfs (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:02:11.373 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package libsnappy1v5:amd64. 2020-04-10T07:02:11.388 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../015-libsnappy1v5_1.1.7-1_amd64.deb ... 2020-04-10T07:02:11.423 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking libsnappy1v5:amd64 (1.1.7-1) ... 2020-04-10T07:02:11.425 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-ceph-common. 2020-04-10T07:02:11.438 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../008-python3-ceph-common_15.2.0-148-g39357d2-1bionic_all.deb ... 2020-04-10T07:02:11.475 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-ceph-common (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:02:11.717 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package libleveldb1v5:amd64. 2020-04-10T07:02:11.732 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../016-libleveldb1v5_1.20-2_amd64.deb ... 2020-04-10T07:02:11.775 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking libleveldb1v5:amd64 (1.20-2) ... 2020-04-10T07:02:11.811 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-prettytable. 2020-04-10T07:02:11.825 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../009-python3-prettytable_0.7.2-3_all.deb ... 2020-04-10T07:02:11.861 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-prettytable (0.7.2-3) ... 2020-04-10T07:02:12.147 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-rbd. 2020-04-10T07:02:12.153 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package liboath0. 2020-04-10T07:02:12.161 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../010-python3-rbd_15.2.0-148-g39357d2-1bionic_amd64.deb ... 2020-04-10T07:02:12.168 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../017-liboath0_2.6.1-1_amd64.deb ... 2020-04-10T07:02:12.197 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-rbd (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:02:12.212 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking liboath0 (2.6.1-1) ... 2020-04-10T07:02:12.506 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package libradosstriper1. 2020-04-10T07:02:12.522 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../018-libradosstriper1_15.2.0-148-g39357d2-1bionic_amd64.deb ... 2020-04-10T07:02:12.561 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking libradosstriper1 (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:02:12.613 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package librabbitmq4:amd64. 2020-04-10T07:02:12.622 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../011-librabbitmq4_0.8.0-1ubuntu0.18.04.2_amd64.deb ... 2020-04-10T07:02:12.667 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2020-04-10T07:02:12.959 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package ceph-common. 2020-04-10T07:02:12.976 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../019-ceph-common_15.2.0-148-g39357d2-1bionic_amd64.deb ... 2020-04-10T07:02:12.989 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package librdkafka1:amd64. 2020-04-10T07:02:13.004 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../012-librdkafka1_0.11.3-1build1_amd64.deb ... 2020-04-10T07:02:13.017 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking ceph-common (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:02:13.047 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking librdkafka1:amd64 (0.11.3-1build1) ... 2020-04-10T07:02:13.450 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package librgw2. 2020-04-10T07:02:13.465 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../013-librgw2_15.2.0-148-g39357d2-1bionic_amd64.deb ... 2020-04-10T07:02:13.508 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking librgw2 (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:02:14.185 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-rgw. 2020-04-10T07:02:14.200 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../014-python3-rgw_15.2.0-148-g39357d2-1bionic_amd64.deb ... 2020-04-10T07:02:14.235 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-rgw (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:02:14.523 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libsnappy1v5:amd64. 2020-04-10T07:02:14.539 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../015-libsnappy1v5_1.1.7-1_amd64.deb ... 2020-04-10T07:02:14.580 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libsnappy1v5:amd64 (1.1.7-1) ... 2020-04-10T07:02:14.764 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package ceph-base. 2020-04-10T07:02:14.777 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../020-ceph-base_15.2.0-148-g39357d2-1bionic_amd64.deb ... 2020-04-10T07:02:14.823 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking ceph-base (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:02:14.899 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libleveldb1v5:amd64. 2020-04-10T07:02:14.913 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../016-libleveldb1v5_1.20-2_amd64.deb ... 2020-04-10T07:02:14.958 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libleveldb1v5:amd64 (1.20-2) ... 2020-04-10T07:02:15.311 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package liboath0. 2020-04-10T07:02:15.324 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../017-liboath0_2.6.1-1_amd64.deb ... 2020-04-10T07:02:15.361 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking liboath0 (2.6.1-1) ... 2020-04-10T07:02:15.630 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libradosstriper1. 2020-04-10T07:02:15.648 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../018-libradosstriper1_15.2.0-148-g39357d2-1bionic_amd64.deb ... 2020-04-10T07:02:15.689 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libradosstriper1 (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:02:15.693 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-dateutil. 2020-04-10T07:02:15.708 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../021-python3-dateutil_2.6.1-1_all.deb ... 2020-04-10T07:02:15.751 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-dateutil (2.6.1-1) ... 2020-04-10T07:02:16.134 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package ceph-common. 2020-04-10T07:02:16.149 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../019-ceph-common_15.2.0-148-g39357d2-1bionic_amd64.deb ... 2020-04-10T07:02:16.192 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking ceph-common (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:02:16.479 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-asn1crypto. 2020-04-10T07:02:16.496 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../022-python3-asn1crypto_0.24.0-1_all.deb ... 2020-04-10T07:02:16.635 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-asn1crypto (0.24.0-1) ... 2020-04-10T07:02:16.824 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-cffi-backend. 2020-04-10T07:02:16.839 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../023-python3-cffi-backend_1.11.5-1_amd64.deb ... 2020-04-10T07:02:16.882 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-cffi-backend (1.11.5-1) ... 2020-04-10T07:02:17.159 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-cryptography. 2020-04-10T07:02:17.175 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../024-python3-cryptography_2.1.4-1ubuntu1.3_amd64.deb ... 2020-04-10T07:02:17.209 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-cryptography (2.1.4-1ubuntu1.3) ... 2020-04-10T07:02:17.504 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-openssl. 2020-04-10T07:02:17.519 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../025-python3-openssl_17.5.0-1ubuntu1_all.deb ... 2020-04-10T07:02:17.554 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-openssl (17.5.0-1ubuntu1) ... 2020-04-10T07:02:17.798 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package ceph-base. 2020-04-10T07:02:17.813 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../020-ceph-base_15.2.0-148-g39357d2-1bionic_amd64.deb ... 2020-04-10T07:02:17.823 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2020-04-10T07:02:17.837 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../026-ceph-mgr-modules-core_15.2.0-148-g39357d2-1bionic_all.deb ... 2020-04-10T07:02:17.857 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking ceph-base (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:02:17.882 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking ceph-mgr-modules-core (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:02:18.253 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-bcrypt. 2020-04-10T07:02:18.266 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../027-python3-bcrypt_3.1.4-2_amd64.deb ... 2020-04-10T07:02:18.303 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-bcrypt (3.1.4-2) ... 2020-04-10T07:02:18.597 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-cherrypy3. 2020-04-10T07:02:18.610 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../028-python3-cherrypy3_8.9.1-2_all.deb ... 2020-04-10T07:02:18.647 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-cherrypy3 (8.9.1-2) ... 2020-04-10T07:02:18.744 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-dateutil. 2020-04-10T07:02:18.758 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../021-python3-dateutil_2.6.1-1_all.deb ... 2020-04-10T07:02:18.811 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-dateutil (2.6.1-1) ... 2020-04-10T07:02:18.992 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-jwt. 2020-04-10T07:02:19.005 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../029-python3-jwt_1.5.3+ds1-1_all.deb ... 2020-04-10T07:02:19.058 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-jwt (1.5.3+ds1-1) ... 2020-04-10T07:02:19.097 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-asn1crypto. 2020-04-10T07:02:19.112 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../022-python3-asn1crypto_0.24.0-1_all.deb ... 2020-04-10T07:02:19.147 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-asn1crypto (0.24.0-1) ... 2020-04-10T07:02:19.378 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-logutils. 2020-04-10T07:02:19.392 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-cffi-backend. 2020-04-10T07:02:19.393 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../030-python3-logutils_0.3.3-5_all.deb ... 2020-04-10T07:02:19.405 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../023-python3-cffi-backend_1.11.5-1_amd64.deb ... 2020-04-10T07:02:19.433 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-cffi-backend (1.11.5-1) ... 2020-04-10T07:02:19.445 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-logutils (0.3.3-5) ... 2020-04-10T07:02:19.747 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-markupsafe. 2020-04-10T07:02:19.761 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../031-python3-markupsafe_1.0-1build1_amd64.deb ... 2020-04-10T07:02:19.778 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-cryptography. 2020-04-10T07:02:19.794 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../024-python3-cryptography_2.1.4-1ubuntu1.3_amd64.deb ... 2020-04-10T07:02:19.805 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-markupsafe (1.0-1build1) ... 2020-04-10T07:02:19.845 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-cryptography (2.1.4-1ubuntu1.3) ... 2020-04-10T07:02:20.109 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-mako. 2020-04-10T07:02:20.122 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../032-python3-mako_1.0.7+ds1-1_all.deb ... 2020-04-10T07:02:20.158 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-mako (1.0.7+ds1-1) ... 2020-04-10T07:02:20.229 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-openssl. 2020-04-10T07:02:20.238 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../025-python3-openssl_17.5.0-1ubuntu1_all.deb ... 2020-04-10T07:02:20.282 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-openssl (17.5.0-1ubuntu1) ... 2020-04-10T07:02:20.486 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-simplegeneric. 2020-04-10T07:02:20.501 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../033-python3-simplegeneric_0.8.1-1_all.deb ... 2020-04-10T07:02:20.543 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2020-04-10T07:02:20.545 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-simplegeneric (0.8.1-1) ... 2020-04-10T07:02:20.557 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../026-ceph-mgr-modules-core_15.2.0-148-g39357d2-1bionic_all.deb ... 2020-04-10T07:02:20.601 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking ceph-mgr-modules-core (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:02:20.839 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-singledispatch. 2020-04-10T07:02:20.853 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../034-python3-singledispatch_3.4.0.3-2_all.deb ... 2020-04-10T07:02:20.897 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-singledispatch (3.4.0.3-2) ... 2020-04-10T07:02:20.914 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-bcrypt. 2020-04-10T07:02:20.930 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../027-python3-bcrypt_3.1.4-2_amd64.deb ... 2020-04-10T07:02:20.973 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-bcrypt (3.1.4-2) ... 2020-04-10T07:02:21.210 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-webob. 2020-04-10T07:02:21.223 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../035-python3-webob_1%3a1.7.3-2fakesync1_all.deb ... 2020-04-10T07:02:21.266 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-webob (1:1.7.3-2fakesync1) ... 2020-04-10T07:02:21.283 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-cherrypy3. 2020-04-10T07:02:21.299 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../028-python3-cherrypy3_8.9.1-2_all.deb ... 2020-04-10T07:02:21.341 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-cherrypy3 (8.9.1-2) ... 2020-04-10T07:02:21.610 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-bs4. 2020-04-10T07:02:21.625 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../036-python3-bs4_4.6.0-1_all.deb ... 2020-04-10T07:02:21.669 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-bs4 (4.6.0-1) ... 2020-04-10T07:02:21.678 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-jwt. 2020-04-10T07:02:21.693 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../029-python3-jwt_1.5.3+ds1-1_all.deb ... 2020-04-10T07:02:21.745 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-jwt (1.5.3+ds1-1) ... 2020-04-10T07:02:21.963 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-waitress. 2020-04-10T07:02:21.979 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../037-python3-waitress_1.0.1-1_all.deb ... 2020-04-10T07:02:22.021 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-waitress (1.0.1-1) ... 2020-04-10T07:02:22.040 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-logutils. 2020-04-10T07:02:22.055 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../030-python3-logutils_0.3.3-5_all.deb ... 2020-04-10T07:02:22.098 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-logutils (0.3.3-5) ... 2020-04-10T07:02:22.341 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-tempita. 2020-04-10T07:02:22.356 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../038-python3-tempita_0.5.2-2_all.deb ... 2020-04-10T07:02:22.399 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-tempita (0.5.2-2) ... 2020-04-10T07:02:22.401 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-markupsafe. 2020-04-10T07:02:22.414 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../031-python3-markupsafe_1.0-1build1_amd64.deb ... 2020-04-10T07:02:22.451 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-markupsafe (1.0-1build1) ... 2020-04-10T07:02:22.702 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-paste. 2020-04-10T07:02:22.716 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../039-python3-paste_2.0.3+dfsg-4ubuntu1_all.deb ... 2020-04-10T07:02:22.760 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-paste (2.0.3+dfsg-4ubuntu1) ... 2020-04-10T07:02:22.796 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-mako. 2020-04-10T07:02:22.810 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../032-python3-mako_1.0.7+ds1-1_all.deb ... 2020-04-10T07:02:22.854 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-mako (1.0.7+ds1-1) ... 2020-04-10T07:02:23.088 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2020-04-10T07:02:23.101 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../040-python-pastedeploy-tpl_1.5.2-4_all.deb ... 2020-04-10T07:02:23.138 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python-pastedeploy-tpl (1.5.2-4) ... 2020-04-10T07:02:23.182 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-simplegeneric. 2020-04-10T07:02:23.198 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../033-python3-simplegeneric_0.8.1-1_all.deb ... 2020-04-10T07:02:23.248 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-simplegeneric (0.8.1-1) ... 2020-04-10T07:02:23.441 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-pastedeploy. 2020-04-10T07:02:23.456 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../041-python3-pastedeploy_1.5.2-4_all.deb ... 2020-04-10T07:02:23.499 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-pastedeploy (1.5.2-4) ... 2020-04-10T07:02:23.643 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-singledispatch. 2020-04-10T07:02:23.657 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../034-python3-singledispatch_3.4.0.3-2_all.deb ... 2020-04-10T07:02:23.701 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-singledispatch (3.4.0.3-2) ... 2020-04-10T07:02:23.802 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-webtest. 2020-04-10T07:02:23.816 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../042-python3-webtest_2.0.28-1ubuntu1_all.deb ... 2020-04-10T07:02:23.860 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-webtest (2.0.28-1ubuntu1) ... 2020-04-10T07:02:24.004 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-webob. 2020-04-10T07:02:24.018 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../035-python3-webob_1%3a1.7.3-2fakesync1_all.deb ... 2020-04-10T07:02:24.063 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-webob (1:1.7.3-2fakesync1) ... 2020-04-10T07:02:24.246 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-pecan. 2020-04-10T07:02:24.260 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../043-python3-pecan_1.2.1-2_all.deb ... 2020-04-10T07:02:24.305 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-pecan (1.2.1-2) ... 2020-04-10T07:02:24.382 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-bs4. 2020-04-10T07:02:24.398 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../036-python3-bs4_4.6.0-1_all.deb ... 2020-04-10T07:02:24.440 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-bs4 (4.6.0-1) ... 2020-04-10T07:02:24.684 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-werkzeug. 2020-04-10T07:02:24.697 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../044-python3-werkzeug_0.14.1+dfsg1-1_all.deb ... 2020-04-10T07:02:24.734 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-werkzeug (0.14.1+dfsg1-1) ... 2020-04-10T07:02:24.760 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-waitress. 2020-04-10T07:02:24.773 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../037-python3-waitress_1.0.1-1_all.deb ... 2020-04-10T07:02:24.810 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-waitress (1.0.1-1) ... 2020-04-10T07:02:25.104 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package ceph-mgr. 2020-04-10T07:02:25.117 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../045-ceph-mgr_15.2.0-148-g39357d2-1bionic_amd64.deb ... 2020-04-10T07:02:25.138 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-tempita. 2020-04-10T07:02:25.153 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../038-python3-tempita_0.5.2-2_all.deb ... 2020-04-10T07:02:25.170 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking ceph-mgr (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:02:25.188 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-tempita (0.5.2-2) ... 2020-04-10T07:02:25.483 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-paste. 2020-04-10T07:02:25.499 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../039-python3-paste_2.0.3+dfsg-4ubuntu1_all.deb ... 2020-04-10T07:02:25.541 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-paste (2.0.3+dfsg-4ubuntu1) ... 2020-04-10T07:02:25.640 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package ceph-mon. 2020-04-10T07:02:25.654 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../046-ceph-mon_15.2.0-148-g39357d2-1bionic_amd64.deb ... 2020-04-10T07:02:25.698 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking ceph-mon (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:02:25.869 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2020-04-10T07:02:25.883 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../040-python-pastedeploy-tpl_1.5.2-4_all.deb ... 2020-04-10T07:02:25.927 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python-pastedeploy-tpl (1.5.2-4) ... 2020-04-10T07:02:26.239 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-pastedeploy. 2020-04-10T07:02:26.253 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../041-python3-pastedeploy_1.5.2-4_all.deb ... 2020-04-10T07:02:26.297 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-pastedeploy (1.5.2-4) ... 2020-04-10T07:02:26.600 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-webtest. 2020-04-10T07:02:26.616 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../042-python3-webtest_2.0.28-1ubuntu1_all.deb ... 2020-04-10T07:02:26.658 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-webtest (2.0.28-1ubuntu1) ... 2020-04-10T07:02:26.995 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-pecan. 2020-04-10T07:02:27.009 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../043-python3-pecan_1.2.1-2_all.deb ... 2020-04-10T07:02:27.053 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-pecan (1.2.1-2) ... 2020-04-10T07:02:27.383 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-werkzeug. 2020-04-10T07:02:27.399 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../044-python3-werkzeug_0.14.1+dfsg1-1_all.deb ... 2020-04-10T07:02:27.463 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-werkzeug (0.14.1+dfsg1-1) ... 2020-04-10T07:02:27.502 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package ceph-osd. 2020-04-10T07:02:27.515 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../047-ceph-osd_15.2.0-148-g39357d2-1bionic_amd64.deb ... 2020-04-10T07:02:27.552 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking ceph-osd (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:02:27.836 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package ceph-mgr. 2020-04-10T07:02:27.850 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../045-ceph-mgr_15.2.0-148-g39357d2-1bionic_amd64.deb ... 2020-04-10T07:02:27.894 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking ceph-mgr (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:02:28.373 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package ceph-mon. 2020-04-10T07:02:28.389 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../046-ceph-mon_15.2.0-148-g39357d2-1bionic_amd64.deb ... 2020-04-10T07:02:28.431 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking ceph-mon (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:02:29.415 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package ceph. 2020-04-10T07:02:29.430 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../048-ceph_15.2.0-148-g39357d2-1bionic_amd64.deb ... 2020-04-10T07:02:29.473 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking ceph (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:02:29.818 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package ceph-fuse. 2020-04-10T07:02:29.832 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../049-ceph-fuse_15.2.0-148-g39357d2-1bionic_amd64.deb ... 2020-04-10T07:02:29.860 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package ceph-osd. 2020-04-10T07:02:29.873 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../047-ceph-osd_15.2.0-148-g39357d2-1bionic_amd64.deb ... 2020-04-10T07:02:29.876 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking ceph-fuse (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:02:29.918 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking ceph-osd (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:02:30.263 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package ceph-mds. 2020-04-10T07:02:30.281 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../050-ceph-mds_15.2.0-148-g39357d2-1bionic_amd64.deb ... 2020-04-10T07:02:30.321 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking ceph-mds (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:02:30.782 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package cephadm. 2020-04-10T07:02:30.798 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../051-cephadm_15.2.0-148-g39357d2-1bionic_amd64.deb ... 2020-04-10T07:02:30.832 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking cephadm (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:02:31.143 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2020-04-10T07:02:31.157 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../052-ceph-mgr-cephadm_15.2.0-148-g39357d2-1bionic_all.deb ... 2020-04-10T07:02:31.201 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking ceph-mgr-cephadm (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:02:31.521 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-repoze.lru. 2020-04-10T07:02:31.537 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../053-python3-repoze.lru_0.7-1_all.deb ... 2020-04-10T07:02:31.579 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-repoze.lru (0.7-1) ... 2020-04-10T07:02:31.824 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package ceph. 2020-04-10T07:02:31.837 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../048-ceph_15.2.0-148-g39357d2-1bionic_amd64.deb ... 2020-04-10T07:02:31.865 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-routes. 2020-04-10T07:02:31.878 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../054-python3-routes_2.4.1-1_all.deb ... 2020-04-10T07:02:31.882 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking ceph (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:02:31.915 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-routes (2.4.1-1) ... 2020-04-10T07:02:32.184 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2020-04-10T07:02:32.185 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package ceph-fuse. 2020-04-10T07:02:32.200 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../049-ceph-fuse_15.2.0-148-g39357d2-1bionic_amd64.deb ... 2020-04-10T07:02:32.201 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../055-ceph-mgr-dashboard_15.2.0-148-g39357d2-1bionic_all.deb ... 2020-04-10T07:02:32.234 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking ceph-mgr-dashboard (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:02:32.243 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking ceph-fuse (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:02:32.625 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package ceph-mds. 2020-04-10T07:02:32.636 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../050-ceph-mds_15.2.0-148-g39357d2-1bionic_amd64.deb ... 2020-04-10T07:02:32.680 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking ceph-mds (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:02:33.101 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package ceph-mgr-diskprediction-cloud. 2020-04-10T07:02:33.105 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../056-ceph-mgr-diskprediction-cloud_15.2.0-148-g39357d2-1bionic_all.deb ... 2020-04-10T07:02:33.149 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking ceph-mgr-diskprediction-cloud (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:02:33.199 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package cephadm. 2020-04-10T07:02:33.215 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../051-cephadm_15.2.0-148-g39357d2-1bionic_amd64.deb ... 2020-04-10T07:02:33.258 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking cephadm (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:02:33.452 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-numpy. 2020-04-10T07:02:33.465 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../057-python3-numpy_1%3a1.13.3-2ubuntu1_amd64.deb ... 2020-04-10T07:02:33.510 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-numpy (1:1.13.3-2ubuntu1) ... 2020-04-10T07:02:33.536 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2020-04-10T07:02:33.551 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../052-ceph-mgr-cephadm_15.2.0-148-g39357d2-1bionic_all.deb ... 2020-04-10T07:02:33.594 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking ceph-mgr-cephadm (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:02:33.939 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-repoze.lru. 2020-04-10T07:02:33.955 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../053-python3-repoze.lru_0.7-1_all.deb ... 2020-04-10T07:02:33.989 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-repoze.lru (0.7-1) ... 2020-04-10T07:02:34.107 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-decorator. 2020-04-10T07:02:34.121 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../058-python3-decorator_4.1.2-1_all.deb ... 2020-04-10T07:02:34.165 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-decorator (4.1.2-1) ... 2020-04-10T07:02:34.300 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-routes. 2020-04-10T07:02:34.316 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../054-python3-routes_2.4.1-1_all.deb ... 2020-04-10T07:02:34.359 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-routes (2.4.1-1) ... 2020-04-10T07:02:34.477 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-scipy. 2020-04-10T07:02:34.492 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../059-python3-scipy_0.19.1-2ubuntu1_amd64.deb ... 2020-04-10T07:02:34.535 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-scipy (0.19.1-2ubuntu1) ... 2020-04-10T07:02:34.669 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2020-04-10T07:02:34.684 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../055-ceph-mgr-dashboard_15.2.0-148-g39357d2-1bionic_all.deb ... 2020-04-10T07:02:34.727 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking ceph-mgr-dashboard (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:02:35.621 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package ceph-mgr-diskprediction-cloud. 2020-04-10T07:02:35.632 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../056-ceph-mgr-diskprediction-cloud_15.2.0-148-g39357d2-1bionic_all.deb ... 2020-04-10T07:02:35.678 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking ceph-mgr-diskprediction-cloud (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:02:35.718 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-sklearn-lib. 2020-04-10T07:02:35.732 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../060-python3-sklearn-lib_0.19.1-3_amd64.deb ... 2020-04-10T07:02:35.777 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-sklearn-lib (0.19.1-3) ... 2020-04-10T07:02:35.988 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-numpy. 2020-04-10T07:02:36.002 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../057-python3-numpy_1%3a1.13.3-2ubuntu1_amd64.deb ... 2020-04-10T07:02:36.047 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-numpy (1:1.13.3-2ubuntu1) ... 2020-04-10T07:02:36.255 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-joblib. 2020-04-10T07:02:36.269 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../061-python3-joblib_0.11-1_all.deb ... 2020-04-10T07:02:36.314 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-joblib (0.11-1) ... 2020-04-10T07:02:36.653 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-decorator. 2020-04-10T07:02:36.666 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../058-python3-decorator_4.1.2-1_all.deb ... 2020-04-10T07:02:36.683 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-sklearn. 2020-04-10T07:02:36.697 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../062-python3-sklearn_0.19.1-3_all.deb ... 2020-04-10T07:02:36.711 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-decorator (4.1.2-1) ... 2020-04-10T07:02:36.741 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-sklearn (0.19.1-3) ... 2020-04-10T07:02:37.056 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-scipy. 2020-04-10T07:02:37.070 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../059-python3-scipy_0.19.1-2ubuntu1_amd64.deb ... 2020-04-10T07:02:37.115 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-scipy (0.19.1-2ubuntu1) ... 2020-04-10T07:02:37.281 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2020-04-10T07:02:37.299 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../063-ceph-mgr-diskprediction-local_15.2.0-148-g39357d2-1bionic_all.deb ... 2020-04-10T07:02:37.345 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking ceph-mgr-diskprediction-local (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:02:38.307 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-sklearn-lib. 2020-04-10T07:02:38.323 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../060-python3-sklearn-lib_0.19.1-3_amd64.deb ... 2020-04-10T07:02:38.365 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-sklearn-lib (0.19.1-3) ... 2020-04-10T07:02:38.409 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-monotonic. 2020-04-10T07:02:38.425 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../064-python3-monotonic_1.1-2_all.deb ... 2020-04-10T07:02:38.467 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-monotonic (1.1-2) ... 2020-04-10T07:02:38.770 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-fasteners. 2020-04-10T07:02:38.786 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../065-python3-fasteners_0.12.0-3_all.deb ... 2020-04-10T07:02:38.828 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-fasteners (0.12.0-3) ... 2020-04-10T07:02:39.131 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-httplib2. 2020-04-10T07:02:39.145 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../066-python3-httplib2_0.9.2+dfsg-1ubuntu0.1_all.deb ... 2020-04-10T07:02:39.189 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-httplib2 (0.9.2+dfsg-1ubuntu0.1) ... 2020-04-10T07:02:39.244 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-joblib. 2020-04-10T07:02:39.258 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../061-python3-joblib_0.11-1_all.deb ... 2020-04-10T07:02:39.302 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-joblib (0.11-1) ... 2020-04-10T07:02:39.492 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-pyasn1. 2020-04-10T07:02:39.505 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../067-python3-pyasn1_0.4.2-3_all.deb ... 2020-04-10T07:02:39.550 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-pyasn1 (0.4.2-3) ... 2020-04-10T07:02:39.614 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-sklearn. 2020-04-10T07:02:39.629 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../062-python3-sklearn_0.19.1-3_all.deb ... 2020-04-10T07:02:39.664 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-sklearn (0.19.1-3) ... 2020-04-10T07:02:39.929 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-pyasn1-modules. 2020-04-10T07:02:39.942 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../068-python3-pyasn1-modules_0.2.1-0.2_all.deb ... 2020-04-10T07:02:39.978 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2) ... 2020-04-10T07:02:40.254 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2020-04-10T07:02:40.267 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../063-ceph-mgr-diskprediction-local_15.2.0-148-g39357d2-1bionic_all.deb ... 2020-04-10T07:02:40.290 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-rsa. 2020-04-10T07:02:40.304 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../069-python3-rsa_3.4.2-1_all.deb ... 2020-04-10T07:02:40.312 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking ceph-mgr-diskprediction-local (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:02:40.347 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-rsa (3.4.2-1) ... 2020-04-10T07:02:40.659 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-sqlalchemy. 2020-04-10T07:02:40.673 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../070-python3-sqlalchemy_1.1.11+ds1-1ubuntu1_all.deb ... 2020-04-10T07:02:40.717 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2020-04-10T07:02:41.164 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-oauth2client. 2020-04-10T07:02:41.177 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../071-python3-oauth2client_4.1.2-3_all.deb ... 2020-04-10T07:02:41.222 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-oauth2client (4.1.2-3) ... 2020-04-10T07:02:41.390 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-monotonic. 2020-04-10T07:02:41.407 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../064-python3-monotonic_1.1-2_all.deb ... 2020-04-10T07:02:41.448 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-monotonic (1.1-2) ... 2020-04-10T07:02:41.541 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-setuptools. 2020-04-10T07:02:41.557 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../072-python3-setuptools_39.0.1-2_all.deb ... 2020-04-10T07:02:41.599 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-setuptools (39.0.1-2) ... 2020-04-10T07:02:41.751 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-fasteners. 2020-04-10T07:02:41.766 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../065-python3-fasteners_0.12.0-3_all.deb ... 2020-04-10T07:02:41.826 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-fasteners (0.12.0-3) ... 2020-04-10T07:02:41.978 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-websocket. 2020-04-10T07:02:41.994 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../073-python3-websocket_0.44.0-0ubuntu2_all.deb ... 2020-04-10T07:02:42.036 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-websocket (0.44.0-0ubuntu2) ... 2020-04-10T07:02:42.104 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-httplib2. 2020-04-10T07:02:42.120 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../066-python3-httplib2_0.9.2+dfsg-1ubuntu0.1_all.deb ... 2020-04-10T07:02:42.154 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-httplib2 (0.9.2+dfsg-1ubuntu0.1) ... 2020-04-10T07:02:42.389 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-kubernetes. 2020-04-10T07:02:42.405 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../074-python3-kubernetes_2.0.0-2ubuntu1_all.deb ... 2020-04-10T07:02:42.456 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-kubernetes (2.0.0-2ubuntu1) ... 2020-04-10T07:02:42.458 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-pyasn1. 2020-04-10T07:02:42.471 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../067-python3-pyasn1_0.4.2-3_all.deb ... 2020-04-10T07:02:42.516 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-pyasn1 (0.4.2-3) ... 2020-04-10T07:02:42.844 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-pyasn1-modules. 2020-04-10T07:02:42.851 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2020-04-10T07:02:42.859 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../068-python3-pyasn1-modules_0.2.1-0.2_all.deb ... 2020-04-10T07:02:42.865 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../075-ceph-mgr-k8sevents_15.2.0-148-g39357d2-1bionic_all.deb ... 2020-04-10T07:02:42.902 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2) ... 2020-04-10T07:02:42.909 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking ceph-mgr-k8sevents (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:02:43.214 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-rsa. 2020-04-10T07:02:43.229 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../069-python3-rsa_3.4.2-1_all.deb ... 2020-04-10T07:02:43.237 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-json-pointer. 2020-04-10T07:02:43.252 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../076-python3-json-pointer_1.10-1_all.deb ... 2020-04-10T07:02:43.272 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-rsa (3.4.2-1) ... 2020-04-10T07:02:43.295 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-json-pointer (1.10-1) ... 2020-04-10T07:02:43.592 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-sqlalchemy. 2020-04-10T07:02:43.607 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../070-python3-sqlalchemy_1.1.11+ds1-1ubuntu1_all.deb ... 2020-04-10T07:02:43.642 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2020-04-10T07:02:43.648 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-jsonpatch. 2020-04-10T07:02:43.665 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../077-python3-jsonpatch_1.19+really1.16-1fakesync1_all.deb ... 2020-04-10T07:02:43.710 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2020-04-10T07:02:44.026 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package ceph-mgr-rook. 2020-04-10T07:02:44.041 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../078-ceph-mgr-rook_15.2.0-148-g39357d2-1bionic_all.deb ... 2020-04-10T07:02:44.055 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-oauth2client. 2020-04-10T07:02:44.070 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../071-python3-oauth2client_4.1.2-3_all.deb ... 2020-04-10T07:02:44.084 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking ceph-mgr-rook (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:02:44.114 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-oauth2client (4.1.2-3) ... 2020-04-10T07:02:44.412 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package libonig4:amd64. 2020-04-10T07:02:44.428 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../079-libonig4_6.7.0-1_amd64.deb ... 2020-04-10T07:02:44.442 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-setuptools. 2020-04-10T07:02:44.457 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../072-python3-setuptools_39.0.1-2_all.deb ... 2020-04-10T07:02:44.470 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking libonig4:amd64 (6.7.0-1) ... 2020-04-10T07:02:44.500 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-setuptools (39.0.1-2) ... 2020-04-10T07:02:44.817 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package libjq1:amd64. 2020-04-10T07:02:44.832 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../080-libjq1_1.5+dfsg-2_amd64.deb ... 2020-04-10T07:02:44.887 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-websocket. 2020-04-10T07:02:44.901 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../073-python3-websocket_0.44.0-0ubuntu2_all.deb ... 2020-04-10T07:02:44.971 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-websocket (0.44.0-0ubuntu2) ... 2020-04-10T07:02:44.977 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking libjq1:amd64 (1.5+dfsg-2) ... 2020-04-10T07:02:45.246 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package jq. 2020-04-10T07:02:45.260 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../081-jq_1.5+dfsg-2_amd64.deb ... 2020-04-10T07:02:45.265 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-kubernetes. 2020-04-10T07:02:45.281 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../074-python3-kubernetes_2.0.0-2ubuntu1_all.deb ... 2020-04-10T07:02:45.304 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking jq (1.5+dfsg-2) ... 2020-04-10T07:02:45.323 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-kubernetes (2.0.0-2ubuntu1) ... 2020-04-10T07:02:45.541 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package socat. 2020-04-10T07:02:45.554 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../082-socat_1.7.3.2-2ubuntu2_amd64.deb ... 2020-04-10T07:02:45.582 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking socat (1.7.3.2-2ubuntu2) ... 2020-04-10T07:02:45.761 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2020-04-10T07:02:45.777 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../075-ceph-mgr-k8sevents_15.2.0-148-g39357d2-1bionic_all.deb ... 2020-04-10T07:02:45.819 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking ceph-mgr-k8sevents (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:02:45.927 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package libxslt1.1:amd64. 2020-04-10T07:02:45.942 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../083-libxslt1.1_1.1.29-5ubuntu0.2_amd64.deb ... 2020-04-10T07:02:45.986 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2020-04-10T07:02:46.138 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-json-pointer. 2020-04-10T07:02:46.154 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../076-python3-json-pointer_1.10-1_all.deb ... 2020-04-10T07:02:46.214 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-json-pointer (1.10-1) ... 2020-04-10T07:02:46.246 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package xmlstarlet. 2020-04-10T07:02:46.263 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../084-xmlstarlet_1.6.1-2_amd64.deb ... 2020-04-10T07:02:46.305 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking xmlstarlet (1.6.1-2) ... 2020-04-10T07:02:46.592 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-jsonpatch. 2020-04-10T07:02:46.606 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../077-python3-jsonpatch_1.19+really1.16-1fakesync1_all.deb ... 2020-04-10T07:02:46.653 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2020-04-10T07:02:46.737 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package ceph-test. 2020-04-10T07:02:46.813 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../085-ceph-test_15.2.0-148-g39357d2-1bionic_amd64.deb ... 2020-04-10T07:02:46.953 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package ceph-mgr-rook. 2020-04-10T07:02:46.970 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../078-ceph-mgr-rook_15.2.0-148-g39357d2-1bionic_all.deb ... 2020-04-10T07:02:47.011 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking ceph-mgr-rook (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:02:47.100 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking ceph-test (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:02:47.324 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libonig4:amd64. 2020-04-10T07:02:47.341 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../079-libonig4_6.7.0-1_amd64.deb ... 2020-04-10T07:02:47.381 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libonig4:amd64 (6.7.0-1) ... 2020-04-10T07:02:47.743 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libjq1:amd64. 2020-04-10T07:02:47.759 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../080-libjq1_1.5+dfsg-2_amd64.deb ... 2020-04-10T07:02:47.878 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libjq1:amd64 (1.5+dfsg-2) ... 2020-04-10T07:02:48.173 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package jq. 2020-04-10T07:02:48.190 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../081-jq_1.5+dfsg-2_amd64.deb ... 2020-04-10T07:02:48.231 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking jq (1.5+dfsg-2) ... 2020-04-10T07:02:48.483 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package socat. 2020-04-10T07:02:48.498 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../082-socat_1.7.3.2-2ubuntu2_amd64.deb ... 2020-04-10T07:02:48.542 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking socat (1.7.3.2-2ubuntu2) ... 2020-04-10T07:02:48.929 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libxslt1.1:amd64. 2020-04-10T07:02:48.949 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../083-libxslt1.1_1.1.29-5ubuntu0.2_amd64.deb ... 2020-04-10T07:02:48.987 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2020-04-10T07:02:49.265 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package xmlstarlet. 2020-04-10T07:02:49.279 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../084-xmlstarlet_1.6.1-2_amd64.deb ... 2020-04-10T07:02:49.323 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking xmlstarlet (1.6.1-2) ... 2020-04-10T07:02:49.669 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package ceph-test. 2020-04-10T07:02:49.685 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../085-ceph-test_15.2.0-148-g39357d2-1bionic_amd64.deb ... 2020-04-10T07:02:49.752 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking ceph-test (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:02:50.378 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package libcephfs-dev. 2020-04-10T07:02:50.392 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../086-libcephfs-dev_15.2.0-148-g39357d2-1bionic_amd64.deb ... 2020-04-10T07:02:50.436 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking libcephfs-dev (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:02:50.722 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python-attr. 2020-04-10T07:02:50.736 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../087-python-attr_17.4.0-2_all.deb ... 2020-04-10T07:02:50.780 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python-attr (17.4.0-2) ... 2020-04-10T07:02:51.067 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python-funcsigs. 2020-04-10T07:02:51.081 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../088-python-funcsigs_1.0.2-4_all.deb ... 2020-04-10T07:02:51.125 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python-funcsigs (1.0.2-4) ... 2020-04-10T07:02:51.427 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python-pluggy. 2020-04-10T07:02:51.443 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../089-python-pluggy_0.6.0-1_all.deb ... 2020-04-10T07:02:51.486 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python-pluggy (0.6.0-1) ... 2020-04-10T07:02:51.788 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python-py. 2020-04-10T07:02:51.802 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../090-python-py_1.5.2-1_all.deb ... 2020-04-10T07:02:51.846 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python-py (1.5.2-1) ... 2020-04-10T07:02:52.175 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python-pytest. 2020-04-10T07:02:52.268 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../091-python-pytest_3.3.2-2_all.deb ... 2020-04-10T07:02:52.269 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python-pytest (3.3.2-2) ... 2020-04-10T07:02:52.570 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-attr. 2020-04-10T07:02:52.587 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../092-python3-attr_17.4.0-2_all.deb ... 2020-04-10T07:02:52.628 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-attr (17.4.0-2) ... 2020-04-10T07:02:52.914 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-cycler. 2020-04-10T07:02:52.931 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../093-python3-cycler_0.10.0-1_all.deb ... 2020-04-10T07:02:52.964 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-cycler (0.10.0-1) ... 2020-04-10T07:02:53.073 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libcephfs-dev. 2020-04-10T07:02:53.089 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../086-libcephfs-dev_15.2.0-148-g39357d2-1bionic_amd64.deb ... 2020-04-10T07:02:53.131 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libcephfs-dev (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:02:53.192 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-webencodings. 2020-04-10T07:02:53.208 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../094-python3-webencodings_0.5-2_all.deb ... 2020-04-10T07:02:53.250 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-webencodings (0.5-2) ... 2020-04-10T07:02:53.426 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python-attr. 2020-04-10T07:02:53.439 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../087-python-attr_17.4.0-2_all.deb ... 2020-04-10T07:02:53.484 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python-attr (17.4.0-2) ... 2020-04-10T07:02:53.569 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-html5lib. 2020-04-10T07:02:53.584 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../095-python3-html5lib_0.999999999-1_all.deb ... 2020-04-10T07:02:53.627 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-html5lib (0.999999999-1) ... 2020-04-10T07:02:53.770 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python-funcsigs. 2020-04-10T07:02:53.787 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../088-python-funcsigs_1.0.2-4_all.deb ... 2020-04-10T07:02:53.828 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python-funcsigs (1.0.2-4) ... 2020-04-10T07:02:53.964 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-lxml:amd64. 2020-04-10T07:02:53.977 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../096-python3-lxml_4.2.1-1ubuntu0.1_amd64.deb ... 2020-04-10T07:02:54.013 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-lxml:amd64 (4.2.1-1ubuntu0.1) ... 2020-04-10T07:02:54.150 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python-pluggy. 2020-04-10T07:02:54.165 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../089-python-pluggy_0.6.0-1_all.deb ... 2020-04-10T07:02:54.213 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python-pluggy (0.6.0-1) ... 2020-04-10T07:02:54.451 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-pyparsing. 2020-04-10T07:02:54.465 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../097-python3-pyparsing_2.2.0+dfsg1-2_all.deb ... 2020-04-10T07:02:54.509 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-pyparsing (2.2.0+dfsg1-2) ... 2020-04-10T07:02:54.536 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python-py. 2020-04-10T07:02:54.551 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../090-python-py_1.5.2-1_all.deb ... 2020-04-10T07:02:54.594 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python-py (1.5.2-1) ... 2020-04-10T07:02:54.845 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-tz. 2020-04-10T07:02:54.861 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../098-python3-tz_2018.3-2_all.deb ... 2020-04-10T07:02:54.903 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-tz (2018.3-2) ... 2020-04-10T07:02:54.905 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python-pytest. 2020-04-10T07:02:54.922 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../091-python-pytest_3.3.2-2_all.deb ... 2020-04-10T07:02:54.965 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python-pytest (3.3.2-2) ... 2020-04-10T07:02:55.222 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-matplotlib. 2020-04-10T07:02:55.236 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../099-python3-matplotlib_2.1.1-2ubuntu3_amd64.deb ... 2020-04-10T07:02:55.281 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-matplotlib (2.1.1-2ubuntu3) ... 2020-04-10T07:02:55.309 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-attr. 2020-04-10T07:02:55.325 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../092-python3-attr_17.4.0-2_all.deb ... 2020-04-10T07:02:55.367 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-attr (17.4.0-2) ... 2020-04-10T07:02:55.670 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-cycler. 2020-04-10T07:02:55.685 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../093-python3-cycler_0.10.0-1_all.deb ... 2020-04-10T07:02:55.728 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-cycler (0.10.0-1) ... 2020-04-10T07:02:55.944 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-olefile. 2020-04-10T07:02:55.960 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../100-python3-olefile_0.45.1-1_all.deb ... 2020-04-10T07:02:56.002 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-olefile (0.45.1-1) ... 2020-04-10T07:02:56.040 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-webencodings. 2020-04-10T07:02:56.056 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../094-python3-webencodings_0.5-2_all.deb ... 2020-04-10T07:02:56.098 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-webencodings (0.5-2) ... 2020-04-10T07:02:56.313 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-pastescript. 2020-04-10T07:02:56.328 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../101-python3-pastescript_2.0.2-2_all.deb ... 2020-04-10T07:02:56.372 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-pastescript (2.0.2-2) ... 2020-04-10T07:02:56.409 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-html5lib. 2020-04-10T07:02:56.423 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../095-python3-html5lib_0.999999999-1_all.deb ... 2020-04-10T07:02:56.467 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-html5lib (0.999999999-1) ... 2020-04-10T07:02:56.725 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-pil:amd64. 2020-04-10T07:02:56.741 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../102-python3-pil_5.1.0-1ubuntu0.2_amd64.deb ... 2020-04-10T07:02:56.792 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-pil:amd64 (5.1.0-1ubuntu0.2) ... 2020-04-10T07:02:56.812 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-lxml:amd64. 2020-04-10T07:02:56.830 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../096-python3-lxml_4.2.1-1ubuntu0.1_amd64.deb ... 2020-04-10T07:02:56.870 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-lxml:amd64 (4.2.1-1ubuntu0.1) ... 2020-04-10T07:02:57.145 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-pluggy. 2020-04-10T07:02:57.162 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../103-python3-pluggy_0.6.0-1_all.deb ... 2020-04-10T07:02:57.220 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-pluggy (0.6.0-1) ... 2020-04-10T07:02:57.299 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-pyparsing. 2020-04-10T07:02:57.314 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../097-python3-pyparsing_2.2.0+dfsg1-2_all.deb ... 2020-04-10T07:02:57.359 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-pyparsing (2.2.0+dfsg1-2) ... 2020-04-10T07:02:57.548 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-py. 2020-04-10T07:02:57.561 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../104-python3-py_1.5.2-1_all.deb ... 2020-04-10T07:02:57.606 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-py (1.5.2-1) ... 2020-04-10T07:02:57.702 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-tz. 2020-04-10T07:02:57.717 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../098-python3-tz_2018.3-2_all.deb ... 2020-04-10T07:02:57.760 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-tz (2018.3-2) ... 2020-04-10T07:02:57.961 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-pyinotify. 2020-04-10T07:02:57.973 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../105-python3-pyinotify_0.9.6-1_all.deb ... 2020-04-10T07:02:58.018 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-pyinotify (0.9.6-1) ... 2020-04-10T07:02:58.080 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-matplotlib. 2020-04-10T07:02:58.094 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../099-python3-matplotlib_2.1.1-2ubuntu3_amd64.deb ... 2020-04-10T07:02:58.138 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-matplotlib (2.1.1-2ubuntu3) ... 2020-04-10T07:02:58.328 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-pytest. 2020-04-10T07:02:58.344 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../106-python3-pytest_3.3.2-2_all.deb ... 2020-04-10T07:02:58.386 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-pytest (3.3.2-2) ... 2020-04-10T07:02:58.740 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-simplejson. 2020-04-10T07:02:58.756 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../107-python3-simplejson_3.13.2-1_amd64.deb ... 2020-04-10T07:02:58.798 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-simplejson (3.13.2-1) ... 2020-04-10T07:02:58.869 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-olefile. 2020-04-10T07:02:58.882 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../100-python3-olefile_0.45.1-1_all.deb ... 2020-04-10T07:02:58.927 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-olefile (0.45.1-1) ... 2020-04-10T07:02:59.126 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package python3-tk:amd64. 2020-04-10T07:02:59.143 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../108-python3-tk_3.6.9-1~18.04_amd64.deb ... 2020-04-10T07:02:59.184 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking python3-tk:amd64 (3.6.9-1~18.04) ... 2020-04-10T07:02:59.223 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-pastescript. 2020-04-10T07:02:59.238 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../101-python3-pastescript_2.0.2-2_all.deb ... 2020-04-10T07:02:59.396 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-pastescript (2.0.2-2) ... 2020-04-10T07:02:59.655 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package radosgw. 2020-04-10T07:02:59.669 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../109-radosgw_15.2.0-148-g39357d2-1bionic_amd64.deb ... 2020-04-10T07:02:59.713 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking radosgw (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:02:59.725 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-pil:amd64. 2020-04-10T07:02:59.740 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../102-python3-pil_5.1.0-1ubuntu0.2_amd64.deb ... 2020-04-10T07:02:59.792 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-pil:amd64 (5.1.0-1ubuntu0.2) ... 2020-04-10T07:03:00.178 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-pluggy. 2020-04-10T07:03:00.191 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../103-python3-pluggy_0.6.0-1_all.deb ... 2020-04-10T07:03:00.236 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-pluggy (0.6.0-1) ... 2020-04-10T07:03:00.556 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-py. 2020-04-10T07:03:00.570 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../104-python3-py_1.5.2-1_all.deb ... 2020-04-10T07:03:00.614 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-py (1.5.2-1) ... 2020-04-10T07:03:00.624 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package rbd-fuse. 2020-04-10T07:03:00.639 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../110-rbd-fuse_15.2.0-148-g39357d2-1bionic_amd64.deb ... 2020-04-10T07:03:00.682 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking rbd-fuse (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:00.960 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-pyinotify. 2020-04-10T07:03:00.973 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../105-python3-pyinotify_0.9.6-1_all.deb ... 2020-04-10T07:03:01.018 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-pyinotify (0.9.6-1) ... 2020-04-10T07:03:01.044 INFO:teuthology.orchestra.run.smithi101.stdout:Selecting previously unselected package rbd-mirror. 2020-04-10T07:03:01.061 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing to unpack .../111-rbd-mirror_15.2.0-148-g39357d2-1bionic_amd64.deb ... 2020-04-10T07:03:01.102 INFO:teuthology.orchestra.run.smithi101.stdout:Unpacking rbd-mirror (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:01.338 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-pytest. 2020-04-10T07:03:01.353 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../106-python3-pytest_3.3.2-2_all.deb ... 2020-04-10T07:03:01.410 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-pytest (3.3.2-2) ... 2020-04-10T07:03:01.618 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-logutils (0.3.3-5) ... 2020-04-10T07:03:01.732 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-simplejson. 2020-04-10T07:03:01.746 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../107-python3-simplejson_3.13.2-1_amd64.deb ... 2020-04-10T07:03:01.790 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-simplejson (3.13.2-1) ... 2020-04-10T07:03:01.924 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-werkzeug (0.14.1+dfsg1-1) ... 2020-04-10T07:03:02.127 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-tk:amd64. 2020-04-10T07:03:02.141 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../108-python3-tk_3.6.9-1~18.04_amd64.deb ... 2020-04-10T07:03:02.185 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-tk:amd64 (3.6.9-1~18.04) ... 2020-04-10T07:03:02.367 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up librdkafka1:amd64 (0.11.3-1build1) ... 2020-04-10T07:03:02.501 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2020-04-10T07:03:02.672 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package radosgw. 2020-04-10T07:03:02.686 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../109-radosgw_15.2.0-148-g39357d2-1bionic_amd64.deb ... 2020-04-10T07:03:02.731 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking radosgw (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:03.129 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-pil:amd64 (5.1.0-1ubuntu0.2) ... 2020-04-10T07:03:03.505 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-simplegeneric (0.8.1-1) ... 2020-04-10T07:03:03.659 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package rbd-fuse. 2020-04-10T07:03:03.673 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../110-rbd-fuse_15.2.0-148-g39357d2-1bionic_amd64.deb ... 2020-04-10T07:03:03.717 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking rbd-fuse (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:03.781 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-webencodings (0.5-2) ... 2020-04-10T07:03:04.066 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python-attr (17.4.0-2) ... 2020-04-10T07:03:04.087 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package rbd-mirror. 2020-04-10T07:03:04.101 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../111-rbd-mirror_15.2.0-148-g39357d2-1bionic_amd64.deb ... 2020-04-10T07:03:04.145 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking rbd-mirror (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:04.267 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-cffi-backend (1.11.5-1) ... 2020-04-10T07:03:04.401 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-py (1.5.2-1) ... 2020-04-10T07:03:04.673 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-logutils (0.3.3-5) ... 2020-04-10T07:03:04.748 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-waitress (1.0.1-1) ... 2020-04-10T07:03:04.976 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-werkzeug (0.14.1+dfsg1-1) ... 2020-04-10T07:03:05.128 INFO:teuthology.orchestra.run.smithi101.stdout:update-alternatives: using /usr/bin/waitress-serve-python3 to provide /usr/bin/waitress-serve (waitress-serve) in auto mode 2020-04-10T07:03:05.335 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up librdkafka1:amd64 (0.11.3-1build1) ... 2020-04-10T07:03:05.422 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-olefile (0.45.1-1) ... 2020-04-10T07:03:05.461 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2020-04-10T07:03:05.732 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-ceph-argparse (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:06.041 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-tempita (0.5.2-2) ... 2020-04-10T07:03:06.097 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-pil:amd64 (5.1.0-1ubuntu0.2) ... 2020-04-10T07:03:06.359 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up socat (1.7.3.2-2ubuntu2) ... 2020-04-10T07:03:06.477 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-numpy (1:1.13.3-2ubuntu1) ... 2020-04-10T07:03:06.482 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-simplegeneric (0.8.1-1) ... 2020-04-10T07:03:06.758 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-webencodings (0.5-2) ... 2020-04-10T07:03:07.051 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python-attr (17.4.0-2) ... 2020-04-10T07:03:07.277 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-cffi-backend (1.11.5-1) ... 2020-04-10T07:03:07.396 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-webob (1:1.7.3-2fakesync1) ... 2020-04-10T07:03:07.403 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-py (1.5.2-1) ... 2020-04-10T07:03:07.706 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-bcrypt (3.1.4-2) ... 2020-04-10T07:03:07.738 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-waitress (1.0.1-1) ... 2020-04-10T07:03:08.008 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python-py (1.5.2-1) ... 2020-04-10T07:03:08.117 INFO:teuthology.orchestra.run.smithi204.stdout:update-alternatives: using /usr/bin/waitress-serve-python3 to provide /usr/bin/waitress-serve (waitress-serve) in auto mode 2020-04-10T07:03:08.233 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-simplejson (3.13.2-1) ... 2020-04-10T07:03:08.408 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-olefile (0.45.1-1) ... 2020-04-10T07:03:08.593 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-json-pointer (1.10-1) ... 2020-04-10T07:03:08.683 INFO:teuthology.orchestra.run.smithi101.stdout:update-alternatives: using /usr/bin/python3-jsonpointer to provide /usr/bin/jsonpointer (jsonpointer) in auto mode 2020-04-10T07:03:08.738 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-ceph-argparse (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:08.936 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up libonig4:amd64 (6.7.0-1) ... 2020-04-10T07:03:09.043 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-tempita (0.5.2-2) ... 2020-04-10T07:03:09.062 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python-pluggy (0.6.0-1) ... 2020-04-10T07:03:09.263 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-singledispatch (3.4.0.3-2) ... 2020-04-10T07:03:09.353 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up socat (1.7.3.2-2ubuntu2) ... 2020-04-10T07:03:09.479 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-numpy (1:1.13.3-2ubuntu1) ... 2020-04-10T07:03:09.564 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-pyparsing (2.2.0+dfsg1-2) ... 2020-04-10T07:03:09.882 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-cycler (0.10.0-1) ... 2020-04-10T07:03:10.158 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-cherrypy3 (8.9.1-2) ... 2020-04-10T07:03:10.535 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-bs4 (4.6.0-1) ... 2020-04-10T07:03:10.707 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-webob (1:1.7.3-2fakesync1) ... 2020-04-10T07:03:10.869 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-pluggy (0.6.0-1) ... 2020-04-10T07:03:11.025 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-bcrypt (3.1.4-2) ... 2020-04-10T07:03:11.162 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up cephadm (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:11.318 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python-py (1.5.2-1) ... 2020-04-10T07:03:11.545 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-simplejson (3.13.2-1) ... 2020-04-10T07:03:11.864 INFO:teuthology.orchestra.run.smithi101.stdout:Adding system user cephadm....done 2020-04-10T07:03:11.921 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-json-pointer (1.10-1) ... 2020-04-10T07:03:12.027 INFO:teuthology.orchestra.run.smithi204.stdout:update-alternatives: using /usr/bin/python3-jsonpointer to provide /usr/bin/jsonpointer (jsonpointer) in auto mode 2020-04-10T07:03:12.035 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-markupsafe (1.0-1build1) ... 2020-04-10T07:03:12.281 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libonig4:amd64 (6.7.0-1) ... 2020-04-10T07:03:12.337 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2020-04-10T07:03:12.408 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python-pluggy (0.6.0-1) ... 2020-04-10T07:03:12.463 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-asn1crypto (0.24.0-1) ... 2020-04-10T07:03:12.617 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-singledispatch (3.4.0.3-2) ... 2020-04-10T07:03:12.822 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-prettytable (0.7.2-3) ... 2020-04-10T07:03:12.918 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-pyparsing (2.2.0+dfsg1-2) ... 2020-04-10T07:03:13.106 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up libjq1:amd64 (1.5+dfsg-2) ... 2020-04-10T07:03:13.233 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-pyasn1 (0.4.2-3) ... 2020-04-10T07:03:13.235 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-cycler (0.10.0-1) ... 2020-04-10T07:03:13.562 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-cherrypy3 (8.9.1-2) ... 2020-04-10T07:03:13.576 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-ceph-common (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:13.902 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-monotonic (1.1-2) ... 2020-04-10T07:03:13.931 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-bs4 (4.6.0-1) ... 2020-04-10T07:03:14.187 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up libsnappy1v5:amd64 (1.1.7-1) ... 2020-04-10T07:03:14.265 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-pluggy (0.6.0-1) ... 2020-04-10T07:03:14.313 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-html5lib (0.999999999-1) ... 2020-04-10T07:03:14.571 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up cephadm (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:14.672 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2020-04-10T07:03:14.762 INFO:teuthology.orchestra.run.smithi101.stdout:update-alternatives: using /usr/bin/python3-jsondiff to provide /usr/bin/jsondiff (jsondiff) in auto mode 2020-04-10T07:03:14.803 INFO:teuthology.orchestra.run.smithi101.stdout:update-alternatives: using /usr/bin/python3-jsonpatch to provide /usr/bin/jsonpatch (jsonpatch) in auto mode 2020-04-10T07:03:15.033 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-repoze.lru (0.7-1) ... 2020-04-10T07:03:15.250 INFO:teuthology.orchestra.run.smithi204.stdout:Adding system user cephadm....done 2020-04-10T07:03:15.342 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-httplib2 (0.9.2+dfsg-1ubuntu0.1) ... 2020-04-10T07:03:15.433 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-markupsafe (1.0-1build1) ... 2020-04-10T07:03:15.660 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2) ... 2020-04-10T07:03:15.732 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2020-04-10T07:03:15.859 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-asn1crypto (0.24.0-1) ... 2020-04-10T07:03:15.986 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-attr (17.4.0-2) ... 2020-04-10T07:03:16.193 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-prettytable (0.7.2-3) ... 2020-04-10T07:03:16.287 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python-funcsigs (1.0.2-4) ... 2020-04-10T07:03:16.486 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libjq1:amd64 (1.5+dfsg-2) ... 2020-04-10T07:03:16.488 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-paste (2.0.3+dfsg-4ubuntu1) ... 2020-04-10T07:03:16.612 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-pyasn1 (0.4.2-3) ... 2020-04-10T07:03:16.907 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-websocket (0.44.0-0ubuntu2) ... 2020-04-10T07:03:16.997 INFO:teuthology.orchestra.run.smithi101.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2020-04-10T07:03:16.997 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-ceph-common (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:17.250 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up liboath0 (2.6.1-1) ... 2020-04-10T07:03:17.323 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-monotonic (1.1-2) ... 2020-04-10T07:03:17.624 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libsnappy1v5:amd64 (1.1.7-1) ... 2020-04-10T07:03:17.751 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-html5lib (0.999999999-1) ... 2020-04-10T07:03:17.860 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2020-04-10T07:03:17.986 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-tk:amd64 (3.6.9-1~18.04) ... 2020-04-10T07:03:18.110 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2020-04-10T07:03:18.200 INFO:teuthology.orchestra.run.smithi204.stdout:update-alternatives: using /usr/bin/python3-jsondiff to provide /usr/bin/jsondiff (jsondiff) in auto mode 2020-04-10T07:03:18.241 INFO:teuthology.orchestra.run.smithi204.stdout:update-alternatives: using /usr/bin/python3-jsonpatch to provide /usr/bin/jsonpatch (jsonpatch) in auto mode 2020-04-10T07:03:18.245 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python-pastedeploy-tpl (1.5.2-4) ... 2020-04-10T07:03:18.371 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-setuptools (39.0.1-2) ... 2020-04-10T07:03:18.487 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-repoze.lru (0.7-1) ... 2020-04-10T07:03:18.756 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up liblttng-ust0:amd64 (2.10.1-1) ... 2020-04-10T07:03:18.788 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-httplib2 (0.9.2+dfsg-1ubuntu0.1) ... 2020-04-10T07:03:18.882 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-decorator (4.1.2-1) ... 2020-04-10T07:03:19.115 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2) ... 2020-04-10T07:03:19.183 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-tz (2018.3-2) ... 2020-04-10T07:03:19.458 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-attr (17.4.0-2) ... 2020-04-10T07:03:19.467 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-uritemplate (0.6-1ubuntu1) ... 2020-04-10T07:03:19.735 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-jwt (1.5.3+ds1-1) ... 2020-04-10T07:03:19.767 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python-funcsigs (1.0.2-4) ... 2020-04-10T07:03:19.985 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-paste (2.0.3+dfsg-4ubuntu1) ... 2020-04-10T07:03:20.045 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-joblib (0.11-1) ... 2020-04-10T07:03:20.395 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-websocket (0.44.0-0ubuntu2) ... 2020-04-10T07:03:20.404 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-routes (2.4.1-1) ... 2020-04-10T07:03:20.485 INFO:teuthology.orchestra.run.smithi204.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2020-04-10T07:03:20.730 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-pyinotify (0.9.6-1) ... 2020-04-10T07:03:20.740 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up liboath0 (2.6.1-1) ... 2020-04-10T07:03:20.932 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2020-04-10T07:03:21.040 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-dateutil (2.6.1-1) ... 2020-04-10T07:03:21.058 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-tk:amd64 (3.6.9-1~18.04) ... 2020-04-10T07:03:21.318 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python-pastedeploy-tpl (1.5.2-4) ... 2020-04-10T07:03:21.366 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-cryptography (2.1.4-1ubuntu1.3) ... 2020-04-10T07:03:21.443 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-setuptools (39.0.1-2) ... 2020-04-10T07:03:21.718 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-fasteners (0.12.0-3) ... 2020-04-10T07:03:21.878 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up liblttng-ust0:amd64 (2.10.1-1) ... 2020-04-10T07:03:22.004 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-decorator (4.1.2-1) ... 2020-04-10T07:03:22.010 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up jq (1.5+dfsg-2) ... 2020-04-10T07:03:22.136 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-scipy (0.19.1-2ubuntu1) ... 2020-04-10T07:03:22.297 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-tz (2018.3-2) ... 2020-04-10T07:03:22.620 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-uritemplate (0.6-1ubuntu1) ... 2020-04-10T07:03:22.916 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-jwt (1.5.3+ds1-1) ... 2020-04-10T07:03:23.235 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-joblib (0.11-1) ... 2020-04-10T07:03:23.317 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-lxml:amd64 (4.2.1-1ubuntu0.1) ... 2020-04-10T07:03:23.618 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-mako (1.0.7+ds1-1) ... 2020-04-10T07:03:23.619 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-routes (2.4.1-1) ... 2020-04-10T07:03:23.929 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-pyinotify (0.9.6-1) ... 2020-04-10T07:03:23.936 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-sklearn-lib (0.19.1-3) ... 2020-04-10T07:03:24.062 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-pytest (3.3.2-2) ... 2020-04-10T07:03:24.238 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-dateutil (2.6.1-1) ... 2020-04-10T07:03:24.430 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python-pytest (3.3.2-2) ... 2020-04-10T07:03:24.564 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-cryptography (2.1.4-1ubuntu1.3) ... 2020-04-10T07:03:24.698 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up libleveldb1v5:amd64 (1.20-2) ... 2020-04-10T07:03:24.832 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up xmlstarlet (1.6.1-2) ... 2020-04-10T07:03:24.950 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up librados2 (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:24.966 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-fasteners (0.12.0-3) ... 2020-04-10T07:03:25.059 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-rsa (3.4.2-1) ... 2020-04-10T07:03:25.276 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up jq (1.5+dfsg-2) ... 2020-04-10T07:03:25.360 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up libcephfs2 (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:25.402 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-scipy (0.19.1-2ubuntu1) ... 2020-04-10T07:03:25.488 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up libcephfs-dev (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:25.612 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-rados (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:25.738 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-openssl (17.5.0-1ubuntu1) ... 2020-04-10T07:03:26.065 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-pastedeploy (1.5.2-4) ... 2020-04-10T07:03:26.373 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-oauth2client (4.1.2-3) ... 2020-04-10T07:03:26.558 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-lxml:amd64 (4.2.1-1ubuntu0.1) ... 2020-04-10T07:03:26.717 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-matplotlib (2.1.1-2ubuntu3) ... 2020-04-10T07:03:26.876 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-mako (1.0.7+ds1-1) ... 2020-04-10T07:03:27.194 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-sklearn-lib (0.19.1-3) ... 2020-04-10T07:03:27.320 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-pytest (3.3.2-2) ... 2020-04-10T07:03:27.544 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-sklearn (0.19.1-3) ... 2020-04-10T07:03:27.696 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python-pytest (3.3.2-2) ... 2020-04-10T07:03:27.964 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libleveldb1v5:amd64 (1.20-2) ... 2020-04-10T07:03:28.099 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up xmlstarlet (1.6.1-2) ... 2020-04-10T07:03:28.225 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up librados2 (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:28.351 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-rsa (3.4.2-1) ... 2020-04-10T07:03:28.427 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-webtest (2.0.28-1ubuntu1) ... 2020-04-10T07:03:28.694 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libcephfs2 (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:28.777 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up libradosstriper1 (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:28.820 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libcephfs-dev (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:28.887 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-cephfs (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:28.946 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-rados (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:29.072 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-openssl (17.5.0-1ubuntu1) ... 2020-04-10T07:03:29.178 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up librgw2 (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:29.304 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up ceph-fuse (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:29.382 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-pastedeploy (1.5.2-4) ... 2020-04-10T07:03:29.515 INFO:teuthology.orchestra.run.smithi101.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2020-04-10T07:03:29.532 INFO:teuthology.orchestra.run.smithi101.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2020-04-10T07:03:29.716 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-oauth2client (4.1.2-3) ... 2020-04-10T07:03:29.781 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up librbd1 (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:29.915 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-pecan (1.2.1-2) ... 2020-04-10T07:03:30.008 INFO:teuthology.orchestra.run.smithi101.stdout:update-alternatives: using /usr/bin/python3-pecan to provide /usr/bin/pecan (pecan) in auto mode 2020-04-10T07:03:30.046 INFO:teuthology.orchestra.run.smithi101.stdout:update-alternatives: using /usr/bin/python3-gunicorn_pecan to provide /usr/bin/gunicorn_pecan (gunicorn_pecan) in auto mode 2020-04-10T07:03:30.109 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-matplotlib (2.1.1-2ubuntu3) ... 2020-04-10T07:03:30.335 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-kubernetes (2.0.0-2ubuntu1) ... 2020-04-10T07:03:30.979 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-sklearn (0.19.1-3) ... 2020-04-10T07:03:30.996 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up ceph-mgr-modules-core (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:31.122 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-pastescript (2.0.2-2) ... 2020-04-10T07:03:31.440 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-rgw (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:31.574 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up rbd-fuse (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:31.700 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up python3-rbd (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:31.809 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up ceph-common (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:31.817 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-webtest (2.0.28-1ubuntu1) ... 2020-04-10T07:03:32.195 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libradosstriper1 (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:32.304 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-cephfs (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:32.335 INFO:teuthology.orchestra.run.smithi101.stdout:Adding group ceph....done 2020-04-10T07:03:32.610 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up librgw2 (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:32.713 INFO:teuthology.orchestra.run.smithi101.stdout:Adding system user ceph....done 2020-04-10T07:03:32.739 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up ceph-fuse (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:32.913 INFO:teuthology.orchestra.run.smithi101.stdout:Setting system user ceph properties....done 2020-04-10T07:03:32.960 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2020-04-10T07:03:32.960 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2020-04-10T07:03:33.074 INFO:teuthology.orchestra.run.smithi101.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2020-04-10T07:03:33.250 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up librbd1 (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:33.289 INFO:teuthology.orchestra.run.smithi101.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2020-04-10T07:03:33.384 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-pecan (1.2.1-2) ... 2020-04-10T07:03:33.474 INFO:teuthology.orchestra.run.smithi204.stdout:update-alternatives: using /usr/bin/python3-pecan to provide /usr/bin/pecan (pecan) in auto mode 2020-04-10T07:03:33.514 INFO:teuthology.orchestra.run.smithi204.stdout:update-alternatives: using /usr/bin/python3-gunicorn_pecan to provide /usr/bin/gunicorn_pecan (gunicorn_pecan) in auto mode 2020-04-10T07:03:33.561 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up radosgw (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:33.829 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-kubernetes (2.0.0-2ubuntu1) ... 2020-04-10T07:03:33.831 INFO:teuthology.orchestra.run.smithi101.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2020-04-10T07:03:33.832 INFO:teuthology.orchestra.run.smithi101.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2020-04-10T07:03:34.230 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up rbd-mirror (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:34.448 INFO:teuthology.orchestra.run.smithi101.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target -> /lib/systemd/system/ceph-rbd-mirror.target. 2020-04-10T07:03:34.448 INFO:teuthology.orchestra.run.smithi101.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target -> /lib/systemd/system/ceph-rbd-mirror.target. 2020-04-10T07:03:34.473 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up ceph-mgr-modules-core (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:34.599 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-pastescript (2.0.2-2) ... 2020-04-10T07:03:34.698 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up ceph-test (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:34.824 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up ceph-base (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:34.934 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-rgw (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:35.060 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up rbd-fuse (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:35.153 INFO:teuthology.orchestra.run.smithi101.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2020-04-10T07:03:35.186 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-rbd (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:35.295 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up ceph-common (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:35.433 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up ceph-mgr (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:35.651 INFO:teuthology.orchestra.run.smithi101.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2020-04-10T07:03:35.651 INFO:teuthology.orchestra.run.smithi101.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2020-04-10T07:03:35.846 INFO:teuthology.orchestra.run.smithi204.stdout:Adding group ceph....done 2020-04-10T07:03:35.960 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up ceph-mgr-diskprediction-local (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:36.094 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up ceph-osd (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:36.249 INFO:teuthology.orchestra.run.smithi204.stdout:Adding system user ceph....done 2020-04-10T07:03:36.425 INFO:teuthology.orchestra.run.smithi204.stdout:Setting system user ceph properties....done 2020-04-10T07:03:36.518 INFO:teuthology.orchestra.run.smithi101.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2020-04-10T07:03:36.518 INFO:teuthology.orchestra.run.smithi101.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2020-04-10T07:03:36.588 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2020-04-10T07:03:36.779 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up ceph-mgr-cephadm (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:36.811 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2020-04-10T07:03:36.922 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up ceph-mgr-diskprediction-cloud (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:37.065 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up ceph-mgr-rook (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:37.072 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up radosgw (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:37.207 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up ceph-mds (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:37.337 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2020-04-10T07:03:37.337 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2020-04-10T07:03:37.430 INFO:teuthology.orchestra.run.smithi101.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2020-04-10T07:03:37.430 INFO:teuthology.orchestra.run.smithi101.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2020-04-10T07:03:37.692 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up ceph-mon (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:37.733 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up rbd-mirror (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:37.907 INFO:teuthology.orchestra.run.smithi101.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2020-04-10T07:03:37.907 INFO:teuthology.orchestra.run.smithi101.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2020-04-10T07:03:37.946 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target -> /lib/systemd/system/ceph-rbd-mirror.target. 2020-04-10T07:03:37.946 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target -> /lib/systemd/system/ceph-rbd-mirror.target. 2020-04-10T07:03:38.195 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up ceph-mgr-k8sevents (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:38.201 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up ceph-test (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:38.312 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up ceph-mgr-dashboard (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:38.327 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up ceph-base (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:38.455 INFO:teuthology.orchestra.run.smithi101.stdout:Setting up ceph (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:38.582 INFO:teuthology.orchestra.run.smithi101.stdout:Processing triggers for systemd (237-3ubuntu10.39) ... 2020-04-10T07:03:38.652 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2020-04-10T07:03:38.799 INFO:teuthology.orchestra.run.smithi101.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2020-04-10T07:03:38.937 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up ceph-mgr (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:39.167 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2020-04-10T07:03:39.167 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2020-04-10T07:03:39.465 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up ceph-mgr-diskprediction-local (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:39.598 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up ceph-osd (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:39.985 INFO:teuthology.orchestra.run.smithi101.stdout:Processing triggers for ureadahead (0.100.0-21) ... 2020-04-10T07:03:40.045 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2020-04-10T07:03:40.046 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2020-04-10T07:03:40.076 INFO:teuthology.orchestra.run.smithi101.stdout:Processing triggers for libc-bin (2.27-3ubuntu1) ... 2020-04-10T07:03:40.333 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up ceph-mgr-cephadm (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:40.478 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up ceph-mgr-diskprediction-cloud (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:40.529 INFO:teuthology.orchestra.run.smithi101.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2020-04-10T07:03:40.531 DEBUG:teuthology.parallel:result is None 2020-04-10T07:03:41.139 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up ceph-mgr-rook (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:41.273 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up ceph-mds (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:41.491 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2020-04-10T07:03:41.491 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2020-04-10T07:03:41.766 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up ceph-mon (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:41.985 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2020-04-10T07:03:41.986 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2020-04-10T07:03:42.276 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up ceph-mgr-k8sevents (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:42.401 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up ceph-mgr-dashboard (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:42.544 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up ceph (15.2.0-148-g39357d2-1bionic) ... 2020-04-10T07:03:42.671 INFO:teuthology.orchestra.run.smithi204.stdout:Processing triggers for systemd (237-3ubuntu10.39) ... 2020-04-10T07:03:42.872 INFO:teuthology.orchestra.run.smithi204.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2020-04-10T07:03:44.032 INFO:teuthology.orchestra.run.smithi204.stdout:Processing triggers for ureadahead (0.100.0-21) ... 2020-04-10T07:03:44.125 INFO:teuthology.orchestra.run.smithi204.stdout:Processing triggers for libc-bin (2.27-3ubuntu1) ... 2020-04-10T07:03:44.565 INFO:teuthology.orchestra.run.smithi204.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2020-04-10T07:03:44.567 DEBUG:teuthology.parallel:result is None 2020-04-10T07:03:44.567 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:03:44.573 INFO:teuthology.orchestra.run.smithi101:> sudo lsb_release -is 2020-04-10T07:03:44.667 INFO:teuthology.orchestra.run.smithi101.stdout:Ubuntu 2020-04-10T07:03:44.673 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-04-10T07:03:44.673 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=39357d2a782a02d1ea09c76cbe7bc6f222a1127a 2020-04-10T07:03:44.923 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:03:44.929 INFO:teuthology.orchestra.run.smithi101:> dpkg-query -W -f '${Version}' ceph 2020-04-10T07:03:44.995 INFO:teuthology.orchestra.run.smithi101.stdout:15.2.0-148-g39357d2-1bionic 2020-04-10T07:03:44.995 INFO:teuthology.packaging:The installed version of ceph is 15.2.0-148-g39357d2-1bionic 2020-04-10T07:03:44.995 INFO:teuthology.task.install:The correct ceph version 15.2.0-148-g39357d2-1bionic is installed. 2020-04-10T07:03:44.996 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:03:45.008 INFO:teuthology.orchestra.run.smithi204:> sudo lsb_release -is 2020-04-10T07:03:45.107 INFO:teuthology.orchestra.run.smithi204.stdout:Ubuntu 2020-04-10T07:03:45.113 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-04-10T07:03:45.113 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=39357d2a782a02d1ea09c76cbe7bc6f222a1127a 2020-04-10T07:03:45.362 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:03:45.368 INFO:teuthology.orchestra.run.smithi204:> dpkg-query -W -f '${Version}' ceph 2020-04-10T07:03:45.433 INFO:teuthology.orchestra.run.smithi204.stdout:15.2.0-148-g39357d2-1bionic 2020-04-10T07:03:45.433 INFO:teuthology.packaging:The installed version of ceph is 15.2.0-148-g39357d2-1bionic 2020-04-10T07:03:45.434 INFO:teuthology.task.install:The correct ceph version 15.2.0-148-g39357d2-1bionic is installed. 2020-04-10T07:03:45.434 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2020-04-10T07:03:45.434 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:03:45.440 INFO:teuthology.orchestra.run.smithi101:> sudo sh -c 'cat > /home/ubuntu/cephtest/valgrind.supp' 2020-04-10T07:03:45.497 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:03:45.503 INFO:teuthology.orchestra.run.smithi204:> sudo sh -c 'cat > /home/ubuntu/cephtest/valgrind.supp' 2020-04-10T07:03:45.563 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2020-04-10T07:03:45.563 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:03:45.568 INFO:teuthology.orchestra.run.smithi101:> sudo sh -c 'cat > /usr/bin/daemon-helper' 2020-04-10T07:03:45.624 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:03:45.670 INFO:teuthology.orchestra.run.smithi101:> sudo chmod a=rx -- /usr/bin/daemon-helper 2020-04-10T07:03:45.729 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:03:45.735 INFO:teuthology.orchestra.run.smithi204:> sudo sh -c 'cat > /usr/bin/daemon-helper' 2020-04-10T07:03:45.794 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:03:45.843 INFO:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/daemon-helper 2020-04-10T07:03:45.905 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2020-04-10T07:03:45.906 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:03:45.911 INFO:teuthology.orchestra.run.smithi101:> sudo sh -c 'cat > /usr/bin/adjust-ulimits' 2020-04-10T07:03:45.999 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:03:46.046 INFO:teuthology.orchestra.run.smithi101:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2020-04-10T07:03:46.103 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:03:46.109 INFO:teuthology.orchestra.run.smithi204:> sudo sh -c 'cat > /usr/bin/adjust-ulimits' 2020-04-10T07:03:46.201 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:03:46.247 INFO:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2020-04-10T07:03:46.305 INFO:teuthology.run_tasks:Running task ceph... 2020-04-10T07:03:46.386 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2020-04-10T07:03:46.386 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:03:46.391 INFO:teuthology.orchestra.run.smithi101:> sudo chmod 777 /var/log/ceph 2020-04-10T07:03:46.435 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:03:46.440 INFO:teuthology.orchestra.run.smithi204:> sudo chmod 777 /var/log/ceph 2020-04-10T07:03:46.496 INFO:tasks.ceph:Disabling ceph logrotate... 2020-04-10T07:03:46.497 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:03:46.502 INFO:teuthology.orchestra.run.smithi101:> sudo rm -f -- /etc/logrotate.d/ceph 2020-04-10T07:03:46.547 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:03:46.552 INFO:teuthology.orchestra.run.smithi204:> sudo rm -f -- /etc/logrotate.d/ceph 2020-04-10T07:03:46.608 INFO:tasks.ceph:Creating extra log directories... 2020-04-10T07:03:46.609 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:03:46.614 INFO:teuthology.orchestra.run.smithi101:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2020-04-10T07:03:46.659 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:03:46.665 INFO:teuthology.orchestra.run.smithi204:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2020-04-10T07:03:46.721 INFO:tasks.ceph:Creating ceph cluster cluster1... 2020-04-10T07:03:46.721 INFO:tasks.ceph:config {'mon_bind_addrvec': True, 'fs': 'xfs', 'log_whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME'], 'skip_mgr_daemons': False, 'cpu_profile': set([]), 'mkfs_options': None, 'cluster': 'cluster1', 'mon_bind_msgr2': True, 'mount_options': None, 'conf': {'client.mirror.5': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.4': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.6': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.1': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.0': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.3': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.2': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'global': {'ms inject socket failures': 5000, 'mon client directed command retry': 5}, 'mgr': {'debug ms': 1, 'debug mgr': 20}, 'client': {'debug rbd': 20, 'rbd default features': 125, 'debug rbd_mirror': 15, 'log to stderr': False}, 'mon': {'debug mon': 20, 'debug paxos': 20, 'debug ms': 1}, 'osd': {'mon osd full ratio': 0.9, 'debug ms': 20, 'bluestore fsck on mount': True, 'debug osd': 25, 'debug bluestore': 20, 'debug bluefs': 20, 'mon osd backfillfull_ratio': 0.85, 'debug rocksdb': 10, 'debug journal': 20, 'mon osd nearfull ratio': 0.8, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'bdev async discard': True, 'bluestore block size': 96636764160, 'debug filestore': 20, 'bdev enable discard': True, 'osd failsafe full ratio': 0.95}}} 2020-04-10T07:03:46.722 INFO:tasks.ceph:ctx.config {'kernel': {'sha1': 'distro', 'kdb': True}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.lock_machines': (2, 'smithi')}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'sha1': 'distro', 'kdb': True}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'sha1': '39357d2a782a02d1ea09c76cbe7bc6f222a1127a', 'extra_packages': ['rbd-mirror']}}, {'ceph': {'cluster': 'cluster1', 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME'], 'fs': 'xfs', 'conf': {'client.mirror.5': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.4': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.6': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.1': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.0': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.3': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.2': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'global': {'ms inject socket failures': 5000, 'mon client directed command retry': 5}, 'mgr': {'debug ms': 1, 'debug mgr': 20}, 'client': {'debug rbd': 20, 'rbd default features': 125, 'debug rbd_mirror': 15, 'log to stderr': False}, 'mon': {'debug mon': 20, 'debug paxos': 20, 'debug ms': 1}, 'osd': {'mon osd full ratio': 0.9, 'debug ms': 20, 'bluestore fsck on mount': True, 'debug osd': 25, 'debug bluestore': 20, 'debug bluefs': 20, 'mon osd backfillfull_ratio': 0.85, 'debug rocksdb': 10, 'debug journal': 20, 'mon osd nearfull ratio': 0.8, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'bdev async discard': True, 'bluestore block size': 96636764160, 'debug filestore': 20, 'bdev enable discard': True, 'osd failsafe full ratio': 0.95}}, 'sha1': '39357d2a782a02d1ea09c76cbe7bc6f222a1127a'}}, {'ceph': {'cluster': 'cluster2'}}, {'exec': {'cluster1.client.mirror.0': ["sudo ceph --cluster cluster1 auth caps client.mirror.4 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster1 auth caps client.mirror.5 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster1 auth caps client.mirror.6 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror.4 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror.5 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror.6 mon 'profile rbd-mirror' osd 'profile rbd'"]}}, {'exec': {'cluster1.client.mirror.0': ["sudo ceph --cluster cluster1 auth caps client.mirror mon 'profile rbd-mirror-peer' osd 'profile rbd'", "sudo ceph --cluster cluster1 auth caps client.mirror.0 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster1 auth caps client.mirror.1 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster1 auth caps client.mirror.2 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster1 auth caps client.mirror.3 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror mon 'profile rbd-mirror-peer' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror.0 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror.1 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror.2 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror.3 mon 'profile rbd-mirror' osd 'profile rbd'"]}}, {'workunit': {'clients': {'cluster1.client.mirror': ['rbd/rbd_mirror_snapshot.sh']}, 'env': {'CEPH_ARGS': '', 'RBD_IMAGE_FEATURES': 'layering,exclusive-lock,journaling', 'RBD_MIRROR_INSTANCES': '4', 'RBD_MIRROR_CONFIG_KEY': '1', 'RBD_MIRROR_USE_EXISTING_CLUSTER': '1'}}}], 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_octopus/qa', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/worker.smithi.2404', 'meta': [{'desc': 'run two ceph clusters and install rbd-mirror'}, {'desc': 'configure the permissions for client.mirror'}, {'desc': 'configure the permissions for client.mirror'}, {'desc': '2 ceph clusters with 1 mon and 3 osds each'}, {'desc': 'run the rbd_mirror_snapshot.sh workunit to test the rbd-mirror daemon'}], 'owner': 'scheduled_teuthology@teuthology', 'openstack': [{'volumes': {'count': 3, 'size': 30}}], 'targets': {'smithi101.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDT9Fklxr7Y5E4UHPwH/a7MBIWYJNi0zKSWKVDcl9NGrTUcSbyXl/q3mNS6XQWwWcfHi4kllR1oWNJMGjcjNMqN2Ik7xQG0CN3j6IxR/ThSBv7eyIUSAldogJUNIte/ae3MyZlpyKhMSxSII/tStu2kQ4lchx7uAYYoXPW8dg178Lr3m/alXSFlfu5TrZIhfa7lKMmYXhXrrkwWDaaX8jqpXi+SYBkAa3mWhHiMihALIKdQRsUmMaAlPYQSnVuA4iU4JIt3x9/0BXHIozs8hrUMQ7MLSjXaKmH3trGj8P/hy65A7yVd1F4RUMUUHcK4AVo/4gHlW/G3+y7vOtBUfDXJ', 'smithi204.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCSzeRmp/sjnId4Ox3+K4SEbMVfVJu6es2vZjpBuMNFfBvRZF3JaOt+F82hMD1lDOJFSzAqT/hKwcdqT5vMPkbkQVnTUTz6KWx5h0Magh8oRggNmVvOswW8Gcz8ktHBV+GSTWy8DISCgruSYDn2z57IoqFYwdf0GFBhZMX1RNs4nqCIHW3RKLCtRlD6g9P2wHktAjGcJk73J8rY66EVc4+hY/V6p67qXU1jU2mCc0ktr4U2vgIRYS4p4AZi1uSYG934vbZTMzvm4YFFt1M6ygHYNPgxF77Z22HIti6AGi3lx0S84WT2wY2NnlAbTeMzvJa8eZnARRhSv+fYJPGVcz+r'}, 'job_id': '4937628', 'tube': 'smithi', 'suite_branch': 'octopus', 'priority': 1000, 'os_version': '18.04', 'first_in_suite': False, 'branch': 'octopus', 'suite': 'rbd', 'email': 'ceph-qa@ceph.io', 'archive_path': '/home/teuthworker/archive/teuthology-2020-04-09_06:00:03-rbd-octopus-distro-basic-smithi/4937628', 'description': 'rbd/mirror/{base/install.yaml clients/{mirror-extra.yaml mirror.yaml} cluster/{2-node.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-low-osd-mem-target.yaml supported-random-distro$/{ubuntu_latest.yaml} workloads/rbd-mirror-snapshot-workunit-journaling.yaml}', 'nuke-on-error': True, 'repo': 'git://git.ceph.com/ceph.git', 'last_in_suite': False, 'machine_type': 'smithi', 'teuthology_branch': 'master', 'sha1': '39357d2a782a02d1ea09c76cbe7bc6f222a1127a', 'name': 'teuthology-2020-04-09_06:00:03-rbd-octopus-distro-basic-smithi', 'roles': [['cluster1.mon.a', 'cluster1.mgr.x', 'cluster2.mgr.x', 'cluster1.osd.0', 'cluster1.osd.1', 'cluster1.osd.2', 'cluster1.client.0', 'cluster2.client.0'], ['cluster2.mon.a', 'cluster2.osd.0', 'cluster2.osd.1', 'cluster2.osd.2', 'cluster1.client.mirror', 'cluster1.client.mirror.0', 'cluster1.client.mirror.1', 'cluster1.client.mirror.2', 'cluster1.client.mirror.3', 'cluster1.client.mirror.4', 'cluster1.client.mirror.5', 'cluster1.client.mirror.6', 'cluster2.client.mirror', 'cluster2.client.mirror.0', 'cluster2.client.mirror.1', 'cluster2.client.mirror.2', 'cluster2.client.mirror.3', 'cluster2.client.mirror.4', 'cluster2.client.mirror.5', 'cluster2.client.mirror.6']], 'suite_relpath': 'qa', 'suite_repo': 'git://git.ceph.com/ceph.git', 'overrides': {'ceph': {'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME'], 'conf': {'client.mirror.5': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.4': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.6': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.1': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.0': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.3': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.2': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'global': {'ms inject socket failures': 5000, 'mon client directed command retry': 5}, 'mgr': {'debug ms': 1, 'debug mgr': 20}, 'client': {'debug rbd': 20, 'rbd default features': 125, 'debug rbd_mirror': 15, 'log to stderr': False}, 'mon': {'debug mon': 20, 'debug paxos': 20, 'debug ms': 1}, 'osd': {'mon osd full ratio': 0.9, 'debug ms': 20, 'bluestore fsck on mount': True, 'debug osd': 25, 'debug bluestore': 20, 'debug bluefs': 20, 'mon osd backfillfull_ratio': 0.85, 'debug rocksdb': 10, 'debug journal': 20, 'mon osd nearfull ratio': 0.8, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'bdev async discard': True, 'bluestore block size': 96636764160, 'debug filestore': 20, 'bdev enable discard': True, 'osd failsafe full ratio': 0.95}}, 'fs': 'xfs', 'sha1': '39357d2a782a02d1ea09c76cbe7bc6f222a1127a'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}}}, 'install': {'ceph': {'sha1': '39357d2a782a02d1ea09c76cbe7bc6f222a1127a'}}, 'admin_socket': {'branch': 'octopus'}, 'thrashosds': {'bdev_inject_crash_probability': 0.5, 'bdev_inject_crash': 2}, 'workunit': {'sha1': '39357d2a782a02d1ea09c76cbe7bc6f222a1127a', 'branch': 'octopus'}}, 'os_type': 'ubuntu', 'suite_sha1': '39357d2a782a02d1ea09c76cbe7bc6f222a1127a'} 2020-04-10T07:03:46.722 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:03:46.729 INFO:teuthology.orchestra.run.smithi101:> install -d -m0755 -- /home/ubuntu/cephtest/cluster1.data 2020-04-10T07:03:46.775 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:03:46.781 INFO:teuthology.orchestra.run.smithi204:> install -d -m0755 -- /home/ubuntu/cephtest/cluster1.data 2020-04-10T07:03:46.833 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:03:46.839 INFO:teuthology.orchestra.run.smithi101:> sudo install -d -m0777 -- /var/run/ceph 2020-04-10T07:03:46.883 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:03:46.888 INFO:teuthology.orchestra.run.smithi204:> sudo install -d -m0777 -- /var/run/ceph 2020-04-10T07:03:46.954 DEBUG:teuthology.orchestra.remote:smithi101:/scratch_devs is 72B 2020-04-10T07:03:46.965 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_4', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_1'] 2020-04-10T07:03:46.965 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:03:46.970 INFO:teuthology.orchestra.run.smithi101:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2020-04-10T07:03:47.020 INFO:teuthology.orchestra.run.smithi101.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-1 2020-04-10T07:03:47.020 INFO:teuthology.orchestra.run.smithi101.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-04-10T07:03:47.021 INFO:teuthology.orchestra.run.smithi101.stdout:Device: 6h/6d Inode: 348 Links: 1 2020-04-10T07:03:47.021 INFO:teuthology.orchestra.run.smithi101.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-04-10T07:03:47.021 INFO:teuthology.orchestra.run.smithi101.stdout:Access: 2020-04-10 07:01:13.731425910 +0000 2020-04-10T07:03:47.021 INFO:teuthology.orchestra.run.smithi101.stdout:Modify: 2020-04-10 07:01:13.615425912 +0000 2020-04-10T07:03:47.022 INFO:teuthology.orchestra.run.smithi101.stdout:Change: 2020-04-10 07:01:13.615425912 +0000 2020-04-10T07:03:47.022 INFO:teuthology.orchestra.run.smithi101.stdout: Birth: - 2020-04-10T07:03:47.030 INFO:teuthology.orchestra.run.smithi101.stderr:1+0 records in 2020-04-10T07:03:47.030 INFO:teuthology.orchestra.run.smithi101.stderr:1+0 records out 2020-04-10T07:03:47.030 INFO:teuthology.orchestra.run.smithi101.stderr:512 bytes copied, 0.000366609 s, 1.4 MB/s 2020-04-10T07:03:47.035 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:03:47.082 INFO:teuthology.orchestra.run.smithi101:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2020-04-10T07:03:47.131 INFO:teuthology.orchestra.run.smithi101.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2020-04-10T07:03:47.132 INFO:teuthology.orchestra.run.smithi101.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-04-10T07:03:47.132 INFO:teuthology.orchestra.run.smithi101.stdout:Device: 6h/6d Inode: 354 Links: 1 2020-04-10T07:03:47.132 INFO:teuthology.orchestra.run.smithi101.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-04-10T07:03:47.132 INFO:teuthology.orchestra.run.smithi101.stdout:Access: 2020-04-10 07:01:14.047425903 +0000 2020-04-10T07:03:47.132 INFO:teuthology.orchestra.run.smithi101.stdout:Modify: 2020-04-10 07:01:13.863425907 +0000 2020-04-10T07:03:47.132 INFO:teuthology.orchestra.run.smithi101.stdout:Change: 2020-04-10 07:01:13.863425907 +0000 2020-04-10T07:03:47.133 INFO:teuthology.orchestra.run.smithi101.stdout: Birth: - 2020-04-10T07:03:47.141 INFO:teuthology.orchestra.run.smithi101.stderr:1+0 records in 2020-04-10T07:03:47.141 INFO:teuthology.orchestra.run.smithi101.stderr:1+0 records out 2020-04-10T07:03:47.141 INFO:teuthology.orchestra.run.smithi101.stderr:512 bytes copied, 0.000319805 s, 1.6 MB/s 2020-04-10T07:03:47.145 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:03:47.194 INFO:teuthology.orchestra.run.smithi101:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2020-04-10T07:03:47.244 INFO:teuthology.orchestra.run.smithi101.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-3 2020-04-10T07:03:47.244 INFO:teuthology.orchestra.run.smithi101.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-04-10T07:03:47.244 INFO:teuthology.orchestra.run.smithi101.stdout:Device: 6h/6d Inode: 360 Links: 1 2020-04-10T07:03:47.244 INFO:teuthology.orchestra.run.smithi101.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-04-10T07:03:47.244 INFO:teuthology.orchestra.run.smithi101.stdout:Access: 2020-04-10 07:01:14.319425897 +0000 2020-04-10T07:03:47.245 INFO:teuthology.orchestra.run.smithi101.stdout:Modify: 2020-04-10 07:01:14.195425899 +0000 2020-04-10T07:03:47.245 INFO:teuthology.orchestra.run.smithi101.stdout:Change: 2020-04-10 07:01:14.195425899 +0000 2020-04-10T07:03:47.245 INFO:teuthology.orchestra.run.smithi101.stdout: Birth: - 2020-04-10T07:03:47.253 INFO:teuthology.orchestra.run.smithi101.stderr:1+0 records in 2020-04-10T07:03:47.253 INFO:teuthology.orchestra.run.smithi101.stderr:1+0 records out 2020-04-10T07:03:47.254 INFO:teuthology.orchestra.run.smithi101.stderr:512 bytes copied, 0.000360662 s, 1.4 MB/s 2020-04-10T07:03:47.258 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:03:47.306 INFO:teuthology.orchestra.run.smithi101:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2020-04-10T07:03:47.356 INFO:teuthology.orchestra.run.smithi101.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-4 2020-04-10T07:03:47.357 INFO:teuthology.orchestra.run.smithi101.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-04-10T07:03:47.357 INFO:teuthology.orchestra.run.smithi101.stdout:Device: 6h/6d Inode: 366 Links: 1 2020-04-10T07:03:47.357 INFO:teuthology.orchestra.run.smithi101.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-04-10T07:03:47.357 INFO:teuthology.orchestra.run.smithi101.stdout:Access: 2020-04-10 07:03:29.574080305 +0000 2020-04-10T07:03:47.357 INFO:teuthology.orchestra.run.smithi101.stdout:Modify: 2020-04-10 07:01:14.451425894 +0000 2020-04-10T07:03:47.358 INFO:teuthology.orchestra.run.smithi101.stdout:Change: 2020-04-10 07:01:14.451425894 +0000 2020-04-10T07:03:47.358 INFO:teuthology.orchestra.run.smithi101.stdout: Birth: - 2020-04-10T07:03:47.365 INFO:teuthology.orchestra.run.smithi101.stderr:1+0 records in 2020-04-10T07:03:47.366 INFO:teuthology.orchestra.run.smithi101.stderr:1+0 records out 2020-04-10T07:03:47.366 INFO:teuthology.orchestra.run.smithi101.stderr:512 bytes copied, 0.000339076 s, 1.5 MB/s 2020-04-10T07:03:47.369 INFO:tasks.ceph:osd dev map: {'cluster1.osd.2': '/dev/vg_nvme/lv_2', 'cluster1.osd.1': '/dev/vg_nvme/lv_3', 'cluster1.osd.0': '/dev/vg_nvme/lv_4'} 2020-04-10T07:03:47.369 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi101.front.sepia.ceph.com'): {'cluster1.osd.2': '/dev/vg_nvme/lv_2', 'cluster1.osd.1': '/dev/vg_nvme/lv_3', 'cluster1.osd.0': '/dev/vg_nvme/lv_4'}} 2020-04-10T07:03:47.370 INFO:tasks.ceph:Generating config... 2020-04-10T07:03:47.371 INFO:tasks.ceph:[client.mirror.5] admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2020-04-10T07:03:47.371 INFO:tasks.ceph:[client.mirror.5] pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2020-04-10T07:03:47.371 INFO:tasks.ceph:[client.mirror.4] admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2020-04-10T07:03:47.371 INFO:tasks.ceph:[client.mirror.4] pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2020-04-10T07:03:47.372 INFO:tasks.ceph:[client.mirror.6] admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2020-04-10T07:03:47.372 INFO:tasks.ceph:[client.mirror.6] pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2020-04-10T07:03:47.372 INFO:tasks.ceph:[client.mirror.1] admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2020-04-10T07:03:47.372 INFO:tasks.ceph:[client.mirror.1] pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2020-04-10T07:03:47.373 INFO:tasks.ceph:[client.mirror.0] admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2020-04-10T07:03:47.373 INFO:tasks.ceph:[client.mirror.0] pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2020-04-10T07:03:47.373 INFO:tasks.ceph:[client.mirror.3] admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2020-04-10T07:03:47.373 INFO:tasks.ceph:[client.mirror.3] pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2020-04-10T07:03:47.373 INFO:tasks.ceph:[client.mirror.2] admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2020-04-10T07:03:47.373 INFO:tasks.ceph:[client.mirror.2] pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2020-04-10T07:03:47.374 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2020-04-10T07:03:47.374 INFO:tasks.ceph:[global] mon client directed command retry = 5 2020-04-10T07:03:47.374 INFO:tasks.ceph:[mgr] debug ms = 1 2020-04-10T07:03:47.374 INFO:tasks.ceph:[mgr] debug mgr = 20 2020-04-10T07:03:47.374 INFO:tasks.ceph:[client] debug rbd = 20 2020-04-10T07:03:47.374 INFO:tasks.ceph:[client] rbd default features = 125 2020-04-10T07:03:47.374 INFO:tasks.ceph:[client] debug rbd_mirror = 15 2020-04-10T07:03:47.375 INFO:tasks.ceph:[client] log to stderr = False 2020-04-10T07:03:47.375 INFO:tasks.ceph:[mon] debug mon = 20 2020-04-10T07:03:47.375 INFO:tasks.ceph:[mon] debug paxos = 20 2020-04-10T07:03:47.375 INFO:tasks.ceph:[mon] debug ms = 1 2020-04-10T07:03:47.375 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2020-04-10T07:03:47.375 INFO:tasks.ceph:[osd] debug ms = 20 2020-04-10T07:03:47.375 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2020-04-10T07:03:47.375 INFO:tasks.ceph:[osd] debug osd = 25 2020-04-10T07:03:47.376 INFO:tasks.ceph:[osd] debug bluestore = 20 2020-04-10T07:03:47.376 INFO:tasks.ceph:[osd] debug bluefs = 20 2020-04-10T07:03:47.376 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2020-04-10T07:03:47.376 INFO:tasks.ceph:[osd] debug rocksdb = 10 2020-04-10T07:03:47.376 INFO:tasks.ceph:[osd] debug journal = 20 2020-04-10T07:03:47.376 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2020-04-10T07:03:47.376 INFO:tasks.ceph:[osd] osd memory target = 1610612736 2020-04-10T07:03:47.376 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2020-04-10T07:03:47.377 INFO:tasks.ceph:[osd] bdev async discard = True 2020-04-10T07:03:47.377 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2020-04-10T07:03:47.378 INFO:tasks.ceph:[osd] debug filestore = 20 2020-04-10T07:03:47.378 INFO:tasks.ceph:[osd] bdev enable discard = True 2020-04-10T07:03:47.378 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2020-04-10T07:03:47.378 INFO:tasks.ceph:Setting up cluster1.mon.a... 2020-04-10T07:03:47.379 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:03:47.419 INFO:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/cluster1.keyring 2020-04-10T07:03:47.496 INFO:teuthology.orchestra.run.smithi101.stdout:creating /etc/ceph/cluster1.keyring 2020-04-10T07:03:47.499 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:03:47.546 INFO:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/cluster1.keyring 2020-04-10T07:03:47.624 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:03:47.670 INFO:teuthology.orchestra.run.smithi101:> sudo chmod 0644 /etc/ceph/cluster1.keyring 2020-04-10T07:03:47.728 DEBUG:tasks.ceph:Ceph mon addresses: [('cluster1.mon.a', '172.21.15.101')] 2020-04-10T07:03:47.728 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:03:47.775 INFO:teuthology.orchestra.run.smithi101:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool --create --clobber --enable-all-features --add a 172.21.15.101 --print /home/ubuntu/cephtest/cluster1.monmap 2020-04-10T07:03:47.845 INFO:teuthology.orchestra.run.smithi101.stdout:monmaptool: monmap file /home/ubuntu/cephtest/cluster1.monmap 2020-04-10T07:03:47.845 INFO:teuthology.orchestra.run.smithi101.stdout:monmaptool: generated fsid 5d7eff56-55fb-4da2-8b7d-3ec54a26cfc2 2020-04-10T07:03:47.845 INFO:teuthology.orchestra.run.smithi101.stdout:epoch 0 2020-04-10T07:03:47.846 INFO:teuthology.orchestra.run.smithi101.stdout:fsid 5d7eff56-55fb-4da2-8b7d-3ec54a26cfc2 2020-04-10T07:03:47.846 INFO:teuthology.orchestra.run.smithi101.stdout:last_changed 2020-04-10T07:03:47.847048+0000 2020-04-10T07:03:47.846 INFO:teuthology.orchestra.run.smithi101.stdout:created 2020-04-10T07:03:47.847048+0000 2020-04-10T07:03:47.846 INFO:teuthology.orchestra.run.smithi101.stdout:min_mon_release 0 (unknown) 2020-04-10T07:03:47.846 INFO:teuthology.orchestra.run.smithi101.stdout:0: [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] mon.a 2020-04-10T07:03:47.846 INFO:teuthology.orchestra.run.smithi101.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/cluster1.monmap (1 monitors) 2020-04-10T07:03:47.847 INFO:tasks.ceph:Writing /etc/ceph/cluster1.conf for FSID 5d7eff56-55fb-4da2-8b7d-3ec54a26cfc2... 2020-04-10T07:03:47.848 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:03:47.894 INFO:teuthology.orchestra.run.smithi101:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/cluster1.conf && sudo chmod 0644 /etc/ceph/cluster1.conf > /dev/null 2020-04-10T07:03:47.939 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:03:47.945 INFO:teuthology.orchestra.run.smithi204:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/cluster1.conf && sudo chmod 0644 /etc/ceph/cluster1.conf > /dev/null 2020-04-10T07:03:47.989 INFO:teuthology.orchestra.run.smithi101.stdout:[global] 2020-04-10T07:03:47.990 INFO:teuthology.orchestra.run.smithi101.stdout: chdir = "" 2020-04-10T07:03:47.990 INFO:teuthology.orchestra.run.smithi101.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-04-10T07:03:47.990 INFO:teuthology.orchestra.run.smithi101.stdout: auth supported = cephx 2020-04-10T07:03:47.990 INFO:teuthology.orchestra.run.smithi101.stdout: 2020-04-10T07:03:47.990 INFO:teuthology.orchestra.run.smithi101.stdout: filestore xattr use omap = true 2020-04-10T07:03:47.991 INFO:teuthology.orchestra.run.smithi101.stdout: 2020-04-10T07:03:47.991 INFO:teuthology.orchestra.run.smithi101.stdout: mon clock drift allowed = 1.000 2020-04-10T07:03:47.991 INFO:teuthology.orchestra.run.smithi101.stdout: 2020-04-10T07:03:47.991 INFO:teuthology.orchestra.run.smithi101.stdout: osd crush chooseleaf type = 0 2020-04-10T07:03:47.991 INFO:teuthology.orchestra.run.smithi101.stdout: auth debug = true 2020-04-10T07:03:47.991 INFO:teuthology.orchestra.run.smithi101.stdout: 2020-04-10T07:03:47.991 INFO:teuthology.orchestra.run.smithi101.stdout: ms die on old message = true 2020-04-10T07:03:47.991 INFO:teuthology.orchestra.run.smithi101.stdout: ms die on bug = true 2020-04-10T07:03:47.992 INFO:teuthology.orchestra.run.smithi101.stdout: 2020-04-10T07:03:47.992 INFO:teuthology.orchestra.run.smithi101.stdout: mon max pg per osd = 10000 # >= luminous 2020-04-10T07:03:47.992 INFO:teuthology.orchestra.run.smithi101.stdout: mon pg warn max object skew = 0 2020-04-10T07:03:47.992 INFO:teuthology.orchestra.run.smithi101.stdout: 2020-04-10T07:03:47.992 INFO:teuthology.orchestra.run.smithi101.stdout: # disable pg_autoscaler by default for new pools 2020-04-10T07:03:47.992 INFO:teuthology.orchestra.run.smithi101.stdout: osd_pool_default_pg_autoscale_mode = off 2020-04-10T07:03:47.993 INFO:teuthology.orchestra.run.smithi101.stdout: 2020-04-10T07:03:47.993 INFO:teuthology.orchestra.run.smithi101.stdout: osd pool default size = 2 2020-04-10T07:03:47.993 INFO:teuthology.orchestra.run.smithi101.stdout: 2020-04-10T07:03:47.993 INFO:teuthology.orchestra.run.smithi101.stdout: mon osd allow primary affinity = true 2020-04-10T07:03:47.993 INFO:teuthology.orchestra.run.smithi101.stdout: mon osd allow pg remap = true 2020-04-10T07:03:47.993 INFO:teuthology.orchestra.run.smithi101.stdout: mon warn on legacy crush tunables = false 2020-04-10T07:03:47.993 INFO:teuthology.orchestra.run.smithi101.stdout: mon warn on crush straw calc version zero = false 2020-04-10T07:03:47.994 INFO:teuthology.orchestra.run.smithi101.stdout: mon warn on no sortbitwise = false 2020-04-10T07:03:47.994 INFO:teuthology.orchestra.run.smithi101.stdout: mon warn on osd down out interval zero = false 2020-04-10T07:03:47.994 INFO:teuthology.orchestra.run.smithi101.stdout: mon warn on too few osds = false 2020-04-10T07:03:47.994 INFO:teuthology.orchestra.run.smithi101.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-04-10T07:03:47.994 INFO:teuthology.orchestra.run.smithi101.stdout: mon_warn_on_pool_no_redundancy = false 2020-04-10T07:03:47.994 INFO:teuthology.orchestra.run.smithi101.stdout: 2020-04-10T07:03:47.994 INFO:teuthology.orchestra.run.smithi101.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-04-10T07:03:47.995 INFO:teuthology.orchestra.run.smithi101.stdout: 2020-04-10T07:03:47.995 INFO:teuthology.orchestra.run.smithi101.stdout: osd default data pool replay window = 5 2020-04-10T07:03:47.995 INFO:teuthology.orchestra.run.smithi101.stdout: 2020-04-10T07:03:47.995 INFO:teuthology.orchestra.run.smithi101.stdout: mon allow pool delete = true 2020-04-10T07:03:47.995 INFO:teuthology.orchestra.run.smithi101.stdout: 2020-04-10T07:03:47.995 INFO:teuthology.orchestra.run.smithi101.stdout: mon cluster log file level = debug 2020-04-10T07:03:47.995 INFO:teuthology.orchestra.run.smithi101.stdout: debug asserts on shutdown = true 2020-04-10T07:03:47.996 INFO:teuthology.orchestra.run.smithi101.stdout: mon host = 172.21.15.101 2020-04-10T07:03:47.996 INFO:teuthology.orchestra.run.smithi101.stdout: ms inject socket failures = 5000 2020-04-10T07:03:47.996 INFO:teuthology.orchestra.run.smithi101.stdout: mon client directed command retry = 5 2020-04-10T07:03:47.996 INFO:teuthology.orchestra.run.smithi101.stdout: fsid = 5d7eff56-55fb-4da2-8b7d-3ec54a26cfc2 2020-04-10T07:03:47.996 INFO:teuthology.orchestra.run.smithi101.stdout: 2020-04-10T07:03:47.996 INFO:teuthology.orchestra.run.smithi101.stdout:[osd] 2020-04-10T07:03:47.996 INFO:teuthology.orchestra.run.smithi101.stdout: osd journal size = 100 2020-04-10T07:03:47.997 INFO:teuthology.orchestra.run.smithi101.stdout: 2020-04-10T07:03:47.997 INFO:teuthology.orchestra.run.smithi101.stdout: osd scrub load threshold = 5.0 2020-04-10T07:03:47.997 INFO:teuthology.orchestra.run.smithi101.stdout: osd scrub max interval = 600 2020-04-10T07:03:47.997 INFO:teuthology.orchestra.run.smithi101.stdout: 2020-04-10T07:03:47.997 INFO:teuthology.orchestra.run.smithi101.stdout: osd recover clone overlap = true 2020-04-10T07:03:47.997 INFO:teuthology.orchestra.run.smithi101.stdout: osd recovery max chunk = 1048576 2020-04-10T07:03:47.997 INFO:teuthology.orchestra.run.smithi101.stdout: 2020-04-10T07:03:47.998 INFO:teuthology.orchestra.run.smithi101.stdout: osd debug shutdown = true 2020-04-10T07:03:47.998 INFO:teuthology.orchestra.run.smithi101.stdout: osd debug op order = true 2020-04-10T07:03:47.998 INFO:teuthology.orchestra.run.smithi101.stdout: osd debug verify stray on activate = true 2020-04-10T07:03:47.998 INFO:teuthology.orchestra.run.smithi101.stdout: 2020-04-10T07:03:47.998 INFO:teuthology.orchestra.run.smithi101.stdout: osd open classes on start = true 2020-04-10T07:03:47.998 INFO:teuthology.orchestra.run.smithi101.stdout: osd debug pg log writeout = true 2020-04-10T07:03:47.998 INFO:teuthology.orchestra.run.smithi101.stdout: 2020-04-10T07:03:47.998 INFO:teuthology.orchestra.run.smithi101.stdout: osd deep scrub update digest min age = 30 2020-04-10T07:03:47.999 INFO:teuthology.orchestra.run.smithi101.stdout: 2020-04-10T07:03:47.999 INFO:teuthology.orchestra.run.smithi101.stdout: osd map max advance = 10 2020-04-10T07:03:47.999 INFO:teuthology.orchestra.run.smithi101.stdout: 2020-04-10T07:03:47.999 INFO:teuthology.orchestra.run.smithi101.stdout: journal zero on create = true 2020-04-10T07:03:47.999 INFO:teuthology.orchestra.run.smithi101.stdout: 2020-04-10T07:03:47.999 INFO:teuthology.orchestra.run.smithi101.stdout: filestore ondisk finisher threads = 3 2020-04-10T07:03:47.999 INFO:teuthology.orchestra.run.smithi101.stdout: filestore apply finisher threads = 3 2020-04-10T07:03:48.000 INFO:teuthology.orchestra.run.smithi101.stdout: 2020-04-10T07:03:48.000 INFO:teuthology.orchestra.run.smithi101.stdout: bdev debug aio = true 2020-04-10T07:03:48.000 INFO:teuthology.orchestra.run.smithi101.stdout: osd debug misdirected ops = true 2020-04-10T07:03:48.000 INFO:teuthology.orchestra.run.smithi101.stdout: mon osd full ratio = 0.9 2020-04-10T07:03:48.000 INFO:teuthology.orchestra.run.smithi101.stdout: debug ms = 20 2020-04-10T07:03:48.000 INFO:teuthology.orchestra.run.smithi101.stdout: bluestore fsck on mount = True 2020-04-10T07:03:48.001 INFO:teuthology.orchestra.run.smithi101.stdout: debug osd = 25 2020-04-10T07:03:48.001 INFO:teuthology.orchestra.run.smithi101.stdout: debug bluestore = 20 2020-04-10T07:03:48.001 INFO:teuthology.orchestra.run.smithi101.stdout: debug bluefs = 20 2020-04-10T07:03:48.001 INFO:teuthology.orchestra.run.smithi101.stdout: mon osd backfillfull_ratio = 0.85 2020-04-10T07:03:48.001 INFO:teuthology.orchestra.run.smithi101.stdout: debug rocksdb = 10 2020-04-10T07:03:48.002 INFO:teuthology.orchestra.run.smithi101.stdout: debug journal = 20 2020-04-10T07:03:48.002 INFO:teuthology.orchestra.run.smithi101.stdout: mon osd nearfull ratio = 0.8 2020-04-10T07:03:48.002 INFO:teuthology.orchestra.run.smithi101.stdout: osd memory target = 1610612736 2020-04-10T07:03:48.002 INFO:teuthology.orchestra.run.smithi101.stdout: osd objectstore = bluestore 2020-04-10T07:03:48.002 INFO:teuthology.orchestra.run.smithi101.stdout: bdev async discard = True 2020-04-10T07:03:48.002 INFO:teuthology.orchestra.run.smithi101.stdout: bluestore block size = 96636764160 2020-04-10T07:03:48.002 INFO:teuthology.orchestra.run.smithi101.stdout: debug filestore = 20 2020-04-10T07:03:48.003 INFO:teuthology.orchestra.run.smithi101.stdout: bdev enable discard = True 2020-04-10T07:03:48.003 INFO:teuthology.orchestra.run.smithi101.stdout: osd failsafe full ratio = 0.95 2020-04-10T07:03:48.003 INFO:teuthology.orchestra.run.smithi101.stdout: 2020-04-10T07:03:48.003 INFO:teuthology.orchestra.run.smithi101.stdout:[mgr] 2020-04-10T07:03:48.003 INFO:teuthology.orchestra.run.smithi101.stdout: debug ms = 1 2020-04-10T07:03:48.003 INFO:teuthology.orchestra.run.smithi101.stdout: debug mgr = 20 2020-04-10T07:03:48.003 INFO:teuthology.orchestra.run.smithi101.stdout: debug mon = 20 2020-04-10T07:03:48.003 INFO:teuthology.orchestra.run.smithi101.stdout: debug auth = 20 2020-04-10T07:03:48.004 INFO:teuthology.orchestra.run.smithi101.stdout: mon reweight min pgs per osd = 4 2020-04-10T07:03:48.004 INFO:teuthology.orchestra.run.smithi101.stdout: mon reweight min bytes per osd = 10 2020-04-10T07:03:48.004 INFO:teuthology.orchestra.run.smithi101.stdout: mgr/telemetry/nag = false 2020-04-10T07:03:48.004 INFO:teuthology.orchestra.run.smithi101.stdout: 2020-04-10T07:03:48.004 INFO:teuthology.orchestra.run.smithi101.stdout:[mon] 2020-04-10T07:03:48.004 INFO:teuthology.orchestra.run.smithi101.stdout: debug ms = 1 2020-04-10T07:03:48.004 INFO:teuthology.orchestra.run.smithi101.stdout: debug mon = 20 2020-04-10T07:03:48.005 INFO:teuthology.orchestra.run.smithi101.stdout: debug paxos = 20 2020-04-10T07:03:48.005 INFO:teuthology.orchestra.run.smithi101.stdout: debug auth = 20 2020-04-10T07:03:48.005 INFO:teuthology.orchestra.run.smithi101.stdout: mon data avail warn = 5 2020-04-10T07:03:48.005 INFO:teuthology.orchestra.run.smithi101.stdout: mon mgr mkfs grace = 240 2020-04-10T07:03:48.005 INFO:teuthology.orchestra.run.smithi101.stdout: mon reweight min pgs per osd = 4 2020-04-10T07:03:48.005 INFO:teuthology.orchestra.run.smithi101.stdout: mon osd reporter subtree level = osd 2020-04-10T07:03:48.005 INFO:teuthology.orchestra.run.smithi101.stdout: mon osd prime pg temp = true 2020-04-10T07:03:48.006 INFO:teuthology.orchestra.run.smithi101.stdout: mon reweight min bytes per osd = 10 2020-04-10T07:03:48.006 INFO:teuthology.orchestra.run.smithi101.stdout: 2020-04-10T07:03:48.006 INFO:teuthology.orchestra.run.smithi101.stdout:[client] 2020-04-10T07:03:48.006 INFO:teuthology.orchestra.run.smithi101.stdout: rgw cache enabled = true 2020-04-10T07:03:48.006 INFO:teuthology.orchestra.run.smithi101.stdout: rgw enable ops log = true 2020-04-10T07:03:48.006 INFO:teuthology.orchestra.run.smithi101.stdout: rgw enable usage log = true 2020-04-10T07:03:48.006 INFO:teuthology.orchestra.run.smithi101.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-04-10T07:03:48.007 INFO:teuthology.orchestra.run.smithi101.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-04-10T07:03:48.007 INFO:teuthology.orchestra.run.smithi101.stdout: debug rbd = 20 2020-04-10T07:03:48.007 INFO:teuthology.orchestra.run.smithi101.stdout: rbd default features = 125 2020-04-10T07:03:48.007 INFO:teuthology.orchestra.run.smithi101.stdout: debug rbd_mirror = 15 2020-04-10T07:03:48.007 INFO:teuthology.orchestra.run.smithi101.stdout: log to stderr = False 2020-04-10T07:03:48.007 INFO:teuthology.orchestra.run.smithi101.stdout:[mon.a] 2020-04-10T07:03:48.007 INFO:teuthology.orchestra.run.smithi101.stdout:[client.mirror.5] 2020-04-10T07:03:48.008 INFO:teuthology.orchestra.run.smithi101.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2020-04-10T07:03:48.008 INFO:teuthology.orchestra.run.smithi101.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2020-04-10T07:03:48.008 INFO:teuthology.orchestra.run.smithi101.stdout:[client.mirror.4] 2020-04-10T07:03:48.008 INFO:teuthology.orchestra.run.smithi101.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2020-04-10T07:03:48.008 INFO:teuthology.orchestra.run.smithi101.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2020-04-10T07:03:48.008 INFO:teuthology.orchestra.run.smithi101.stdout:[client.mirror.6] 2020-04-10T07:03:48.008 INFO:teuthology.orchestra.run.smithi101.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2020-04-10T07:03:48.009 INFO:teuthology.orchestra.run.smithi101.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2020-04-10T07:03:48.009 INFO:teuthology.orchestra.run.smithi101.stdout:[client.mirror.1] 2020-04-10T07:03:48.009 INFO:teuthology.orchestra.run.smithi101.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2020-04-10T07:03:48.009 INFO:teuthology.orchestra.run.smithi101.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2020-04-10T07:03:48.009 INFO:teuthology.orchestra.run.smithi101.stdout:[client.mirror.0] 2020-04-10T07:03:48.009 INFO:teuthology.orchestra.run.smithi101.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2020-04-10T07:03:48.009 INFO:teuthology.orchestra.run.smithi101.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2020-04-10T07:03:48.010 INFO:teuthology.orchestra.run.smithi101.stdout:[client.mirror.3] 2020-04-10T07:03:48.010 INFO:teuthology.orchestra.run.smithi101.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2020-04-10T07:03:48.010 INFO:teuthology.orchestra.run.smithi101.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2020-04-10T07:03:48.010 INFO:teuthology.orchestra.run.smithi101.stdout:[client.mirror.2] 2020-04-10T07:03:48.010 INFO:teuthology.orchestra.run.smithi101.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2020-04-10T07:03:48.010 INFO:teuthology.orchestra.run.smithi101.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2020-04-10T07:03:48.021 INFO:teuthology.orchestra.run.smithi204.stdout:[global] 2020-04-10T07:03:48.021 INFO:teuthology.orchestra.run.smithi204.stdout: chdir = "" 2020-04-10T07:03:48.021 INFO:teuthology.orchestra.run.smithi204.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-04-10T07:03:48.021 INFO:teuthology.orchestra.run.smithi204.stdout: auth supported = cephx 2020-04-10T07:03:48.022 INFO:teuthology.orchestra.run.smithi204.stdout: 2020-04-10T07:03:48.022 INFO:teuthology.orchestra.run.smithi204.stdout: filestore xattr use omap = true 2020-04-10T07:03:48.022 INFO:teuthology.orchestra.run.smithi204.stdout: 2020-04-10T07:03:48.022 INFO:teuthology.orchestra.run.smithi204.stdout: mon clock drift allowed = 1.000 2020-04-10T07:03:48.022 INFO:teuthology.orchestra.run.smithi204.stdout: 2020-04-10T07:03:48.023 INFO:teuthology.orchestra.run.smithi204.stdout: osd crush chooseleaf type = 0 2020-04-10T07:03:48.023 INFO:teuthology.orchestra.run.smithi204.stdout: auth debug = true 2020-04-10T07:03:48.023 INFO:teuthology.orchestra.run.smithi204.stdout: 2020-04-10T07:03:48.023 INFO:teuthology.orchestra.run.smithi204.stdout: ms die on old message = true 2020-04-10T07:03:48.023 INFO:teuthology.orchestra.run.smithi204.stdout: ms die on bug = true 2020-04-10T07:03:48.023 INFO:teuthology.orchestra.run.smithi204.stdout: 2020-04-10T07:03:48.023 INFO:teuthology.orchestra.run.smithi204.stdout: mon max pg per osd = 10000 # >= luminous 2020-04-10T07:03:48.024 INFO:teuthology.orchestra.run.smithi204.stdout: mon pg warn max object skew = 0 2020-04-10T07:03:48.024 INFO:teuthology.orchestra.run.smithi204.stdout: 2020-04-10T07:03:48.024 INFO:teuthology.orchestra.run.smithi204.stdout: # disable pg_autoscaler by default for new pools 2020-04-10T07:03:48.024 INFO:teuthology.orchestra.run.smithi204.stdout: osd_pool_default_pg_autoscale_mode = off 2020-04-10T07:03:48.024 INFO:teuthology.orchestra.run.smithi204.stdout: 2020-04-10T07:03:48.024 INFO:teuthology.orchestra.run.smithi204.stdout: osd pool default size = 2 2020-04-10T07:03:48.024 INFO:teuthology.orchestra.run.smithi204.stdout: 2020-04-10T07:03:48.025 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd allow primary affinity = true 2020-04-10T07:03:48.025 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd allow pg remap = true 2020-04-10T07:03:48.025 INFO:teuthology.orchestra.run.smithi204.stdout: mon warn on legacy crush tunables = false 2020-04-10T07:03:48.025 INFO:teuthology.orchestra.run.smithi204.stdout: mon warn on crush straw calc version zero = false 2020-04-10T07:03:48.025 INFO:teuthology.orchestra.run.smithi204.stdout: mon warn on no sortbitwise = false 2020-04-10T07:03:48.025 INFO:teuthology.orchestra.run.smithi204.stdout: mon warn on osd down out interval zero = false 2020-04-10T07:03:48.025 INFO:teuthology.orchestra.run.smithi204.stdout: mon warn on too few osds = false 2020-04-10T07:03:48.025 INFO:teuthology.orchestra.run.smithi204.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-04-10T07:03:48.026 INFO:teuthology.orchestra.run.smithi204.stdout: mon_warn_on_pool_no_redundancy = false 2020-04-10T07:03:48.026 INFO:teuthology.orchestra.run.smithi204.stdout: 2020-04-10T07:03:48.026 INFO:teuthology.orchestra.run.smithi204.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-04-10T07:03:48.026 INFO:teuthology.orchestra.run.smithi204.stdout: 2020-04-10T07:03:48.026 INFO:teuthology.orchestra.run.smithi204.stdout: osd default data pool replay window = 5 2020-04-10T07:03:48.026 INFO:teuthology.orchestra.run.smithi204.stdout: 2020-04-10T07:03:48.026 INFO:teuthology.orchestra.run.smithi204.stdout: mon allow pool delete = true 2020-04-10T07:03:48.027 INFO:teuthology.orchestra.run.smithi204.stdout: 2020-04-10T07:03:48.027 INFO:teuthology.orchestra.run.smithi204.stdout: mon cluster log file level = debug 2020-04-10T07:03:48.027 INFO:teuthology.orchestra.run.smithi204.stdout: debug asserts on shutdown = true 2020-04-10T07:03:48.027 INFO:teuthology.orchestra.run.smithi204.stdout: mon host = 172.21.15.101 2020-04-10T07:03:48.027 INFO:teuthology.orchestra.run.smithi204.stdout: ms inject socket failures = 5000 2020-04-10T07:03:48.027 INFO:teuthology.orchestra.run.smithi204.stdout: mon client directed command retry = 5 2020-04-10T07:03:48.027 INFO:teuthology.orchestra.run.smithi204.stdout: fsid = 5d7eff56-55fb-4da2-8b7d-3ec54a26cfc2 2020-04-10T07:03:48.028 INFO:teuthology.orchestra.run.smithi204.stdout: 2020-04-10T07:03:48.028 INFO:teuthology.orchestra.run.smithi204.stdout:[osd] 2020-04-10T07:03:48.028 INFO:teuthology.orchestra.run.smithi204.stdout: osd journal size = 100 2020-04-10T07:03:48.028 INFO:teuthology.orchestra.run.smithi204.stdout: 2020-04-10T07:03:48.028 INFO:teuthology.orchestra.run.smithi204.stdout: osd scrub load threshold = 5.0 2020-04-10T07:03:48.028 INFO:teuthology.orchestra.run.smithi204.stdout: osd scrub max interval = 600 2020-04-10T07:03:48.028 INFO:teuthology.orchestra.run.smithi204.stdout: 2020-04-10T07:03:48.029 INFO:teuthology.orchestra.run.smithi204.stdout: osd recover clone overlap = true 2020-04-10T07:03:48.029 INFO:teuthology.orchestra.run.smithi204.stdout: osd recovery max chunk = 1048576 2020-04-10T07:03:48.029 INFO:teuthology.orchestra.run.smithi204.stdout: 2020-04-10T07:03:48.029 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug shutdown = true 2020-04-10T07:03:48.029 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug op order = true 2020-04-10T07:03:48.029 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug verify stray on activate = true 2020-04-10T07:03:48.029 INFO:teuthology.orchestra.run.smithi204.stdout: 2020-04-10T07:03:48.029 INFO:teuthology.orchestra.run.smithi204.stdout: osd open classes on start = true 2020-04-10T07:03:48.030 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug pg log writeout = true 2020-04-10T07:03:48.030 INFO:teuthology.orchestra.run.smithi204.stdout: 2020-04-10T07:03:48.030 INFO:teuthology.orchestra.run.smithi204.stdout: osd deep scrub update digest min age = 30 2020-04-10T07:03:48.030 INFO:teuthology.orchestra.run.smithi204.stdout: 2020-04-10T07:03:48.030 INFO:teuthology.orchestra.run.smithi204.stdout: osd map max advance = 10 2020-04-10T07:03:48.030 INFO:teuthology.orchestra.run.smithi204.stdout: 2020-04-10T07:03:48.030 INFO:teuthology.orchestra.run.smithi204.stdout: journal zero on create = true 2020-04-10T07:03:48.031 INFO:teuthology.orchestra.run.smithi204.stdout: 2020-04-10T07:03:48.031 INFO:teuthology.orchestra.run.smithi204.stdout: filestore ondisk finisher threads = 3 2020-04-10T07:03:48.031 INFO:teuthology.orchestra.run.smithi204.stdout: filestore apply finisher threads = 3 2020-04-10T07:03:48.031 INFO:teuthology.orchestra.run.smithi204.stdout: 2020-04-10T07:03:48.031 INFO:teuthology.orchestra.run.smithi204.stdout: bdev debug aio = true 2020-04-10T07:03:48.031 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug misdirected ops = true 2020-04-10T07:03:48.031 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd full ratio = 0.9 2020-04-10T07:03:48.032 INFO:teuthology.orchestra.run.smithi204.stdout: debug ms = 20 2020-04-10T07:03:48.032 INFO:teuthology.orchestra.run.smithi204.stdout: bluestore fsck on mount = True 2020-04-10T07:03:48.032 INFO:teuthology.orchestra.run.smithi204.stdout: debug osd = 25 2020-04-10T07:03:48.032 INFO:teuthology.orchestra.run.smithi204.stdout: debug bluestore = 20 2020-04-10T07:03:48.032 INFO:teuthology.orchestra.run.smithi204.stdout: debug bluefs = 20 2020-04-10T07:03:48.033 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd backfillfull_ratio = 0.85 2020-04-10T07:03:48.033 INFO:teuthology.orchestra.run.smithi204.stdout: debug rocksdb = 10 2020-04-10T07:03:48.033 INFO:teuthology.orchestra.run.smithi204.stdout: debug journal = 20 2020-04-10T07:03:48.033 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd nearfull ratio = 0.8 2020-04-10T07:03:48.033 INFO:teuthology.orchestra.run.smithi204.stdout: osd memory target = 1610612736 2020-04-10T07:03:48.033 INFO:teuthology.orchestra.run.smithi204.stdout: osd objectstore = bluestore 2020-04-10T07:03:48.033 INFO:teuthology.orchestra.run.smithi204.stdout: bdev async discard = True 2020-04-10T07:03:48.034 INFO:teuthology.orchestra.run.smithi204.stdout: bluestore block size = 96636764160 2020-04-10T07:03:48.034 INFO:teuthology.orchestra.run.smithi204.stdout: debug filestore = 20 2020-04-10T07:03:48.034 INFO:teuthology.orchestra.run.smithi204.stdout: bdev enable discard = True 2020-04-10T07:03:48.034 INFO:teuthology.orchestra.run.smithi204.stdout: osd failsafe full ratio = 0.95 2020-04-10T07:03:48.034 INFO:teuthology.orchestra.run.smithi204.stdout: 2020-04-10T07:03:48.034 INFO:teuthology.orchestra.run.smithi204.stdout:[mgr] 2020-04-10T07:03:48.034 INFO:teuthology.orchestra.run.smithi204.stdout: debug ms = 1 2020-04-10T07:03:48.035 INFO:teuthology.orchestra.run.smithi204.stdout: debug mgr = 20 2020-04-10T07:03:48.035 INFO:teuthology.orchestra.run.smithi204.stdout: debug mon = 20 2020-04-10T07:03:48.035 INFO:teuthology.orchestra.run.smithi204.stdout: debug auth = 20 2020-04-10T07:03:48.035 INFO:teuthology.orchestra.run.smithi204.stdout: mon reweight min pgs per osd = 4 2020-04-10T07:03:48.035 INFO:teuthology.orchestra.run.smithi204.stdout: mon reweight min bytes per osd = 10 2020-04-10T07:03:48.035 INFO:teuthology.orchestra.run.smithi204.stdout: mgr/telemetry/nag = false 2020-04-10T07:03:48.035 INFO:teuthology.orchestra.run.smithi204.stdout: 2020-04-10T07:03:48.036 INFO:teuthology.orchestra.run.smithi204.stdout:[mon] 2020-04-10T07:03:48.036 INFO:teuthology.orchestra.run.smithi204.stdout: debug ms = 1 2020-04-10T07:03:48.036 INFO:teuthology.orchestra.run.smithi204.stdout: debug mon = 20 2020-04-10T07:03:48.036 INFO:teuthology.orchestra.run.smithi204.stdout: debug paxos = 20 2020-04-10T07:03:48.036 INFO:teuthology.orchestra.run.smithi204.stdout: debug auth = 20 2020-04-10T07:03:48.041 INFO:teuthology.orchestra.run.smithi204.stdout: mon data avail warn = 5 2020-04-10T07:03:48.042 INFO:teuthology.orchestra.run.smithi204.stdout: mon mgr mkfs grace = 240 2020-04-10T07:03:48.042 INFO:teuthology.orchestra.run.smithi204.stdout: mon reweight min pgs per osd = 4 2020-04-10T07:03:48.042 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd reporter subtree level = osd 2020-04-10T07:03:48.042 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd prime pg temp = true 2020-04-10T07:03:48.042 INFO:teuthology.orchestra.run.smithi204.stdout: mon reweight min bytes per osd = 10 2020-04-10T07:03:48.043 INFO:teuthology.orchestra.run.smithi204.stdout: 2020-04-10T07:03:48.043 INFO:teuthology.orchestra.run.smithi204.stdout:[client] 2020-04-10T07:03:48.043 INFO:teuthology.orchestra.run.smithi204.stdout: rgw cache enabled = true 2020-04-10T07:03:48.043 INFO:teuthology.orchestra.run.smithi204.stdout: rgw enable ops log = true 2020-04-10T07:03:48.043 INFO:teuthology.orchestra.run.smithi204.stdout: rgw enable usage log = true 2020-04-10T07:03:48.043 INFO:teuthology.orchestra.run.smithi204.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-04-10T07:03:48.043 INFO:teuthology.orchestra.run.smithi204.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-04-10T07:03:48.044 INFO:teuthology.orchestra.run.smithi204.stdout: debug rbd = 20 2020-04-10T07:03:48.044 INFO:teuthology.orchestra.run.smithi204.stdout: rbd default features = 125 2020-04-10T07:03:48.044 INFO:teuthology.orchestra.run.smithi204.stdout: debug rbd_mirror = 15 2020-04-10T07:03:48.044 INFO:teuthology.orchestra.run.smithi204.stdout: log to stderr = False 2020-04-10T07:03:48.044 INFO:teuthology.orchestra.run.smithi204.stdout:[mon.a] 2020-04-10T07:03:48.044 INFO:teuthology.orchestra.run.smithi204.stdout:[client.mirror.5] 2020-04-10T07:03:48.045 INFO:teuthology.orchestra.run.smithi204.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2020-04-10T07:03:48.045 INFO:teuthology.orchestra.run.smithi204.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2020-04-10T07:03:48.045 INFO:teuthology.orchestra.run.smithi204.stdout:[client.mirror.4] 2020-04-10T07:03:48.045 INFO:teuthology.orchestra.run.smithi204.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2020-04-10T07:03:48.045 INFO:teuthology.orchestra.run.smithi204.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2020-04-10T07:03:48.045 INFO:teuthology.orchestra.run.smithi204.stdout:[client.mirror.6] 2020-04-10T07:03:48.045 INFO:teuthology.orchestra.run.smithi204.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2020-04-10T07:03:48.046 INFO:teuthology.orchestra.run.smithi204.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2020-04-10T07:03:48.046 INFO:teuthology.orchestra.run.smithi204.stdout:[client.mirror.1] 2020-04-10T07:03:48.046 INFO:teuthology.orchestra.run.smithi204.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2020-04-10T07:03:48.046 INFO:teuthology.orchestra.run.smithi204.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2020-04-10T07:03:48.046 INFO:teuthology.orchestra.run.smithi204.stdout:[client.mirror.0] 2020-04-10T07:03:48.046 INFO:teuthology.orchestra.run.smithi204.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2020-04-10T07:03:48.046 INFO:teuthology.orchestra.run.smithi204.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2020-04-10T07:03:48.047 INFO:teuthology.orchestra.run.smithi204.stdout:[client.mirror.3] 2020-04-10T07:03:48.047 INFO:teuthology.orchestra.run.smithi204.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2020-04-10T07:03:48.047 INFO:teuthology.orchestra.run.smithi204.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2020-04-10T07:03:48.047 INFO:teuthology.orchestra.run.smithi204.stdout:[client.mirror.2] 2020-04-10T07:03:48.047 INFO:teuthology.orchestra.run.smithi204.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2020-04-10T07:03:48.047 INFO:teuthology.orchestra.run.smithi204.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2020-04-10T07:03:48.048 INFO:tasks.ceph:Creating admin key on cluster1.mon.a... 2020-04-10T07:03:48.048 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:03:48.058 INFO:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/cluster1.keyring 2020-04-10T07:03:48.152 INFO:tasks.ceph:Copying monmap to all nodes... 2020-04-10T07:03:48.201 DEBUG:teuthology.orchestra.remote:smithi101:/etc/ceph/cluster1.keyring is 206B 2020-04-10T07:03:48.219 DEBUG:teuthology.orchestra.remote:smithi101:/home/ubuntu/cephtest/cluster1.monmap is 179B 2020-04-10T07:03:48.271 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi101.front.sepia.ceph.com 2020-04-10T07:03:48.271 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:03:48.277 INFO:teuthology.orchestra.run.smithi101:> sudo sh -c 'cat > /etc/ceph/cluster1.keyring' && sudo chmod 0644 /etc/ceph/cluster1.keyring 2020-04-10T07:03:48.373 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:03:48.422 INFO:teuthology.orchestra.run.smithi101:> cat > /home/ubuntu/cephtest/cluster1.monmap 2020-04-10T07:03:48.511 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi204.front.sepia.ceph.com 2020-04-10T07:03:48.511 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:03:48.516 INFO:teuthology.orchestra.run.smithi204:> sudo sh -c 'cat > /etc/ceph/cluster1.keyring' && sudo chmod 0644 /etc/ceph/cluster1.keyring 2020-04-10T07:03:48.614 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:03:48.660 INFO:teuthology.orchestra.run.smithi204:> cat > /home/ubuntu/cephtest/cluster1.monmap 2020-04-10T07:03:48.748 INFO:tasks.ceph:Setting up mon nodes... 2020-04-10T07:03:48.748 INFO:tasks.ceph:Setting up mgr nodes... 2020-04-10T07:03:48.749 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:03:48.754 INFO:teuthology.orchestra.run.smithi101:> sudo mkdir -p /var/lib/ceph/mgr/cluster1-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/cluster1-x/keyring 2020-04-10T07:03:48.838 INFO:teuthology.orchestra.run.smithi101.stdout:creating /var/lib/ceph/mgr/cluster1-x/keyring 2020-04-10T07:03:48.840 INFO:tasks.ceph:Setting up mds nodes... 2020-04-10T07:03:48.840 INFO:tasks.ceph_client:Setting up client nodes... 2020-04-10T07:03:48.840 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:03:48.886 INFO:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/cluster1.client.0.keyring && sudo chmod 0644 /etc/ceph/cluster1.client.0.keyring 2020-04-10T07:03:48.964 INFO:teuthology.orchestra.run.smithi101.stdout:creating /etc/ceph/cluster1.client.0.keyring 2020-04-10T07:03:48.974 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:03:48.981 INFO:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror /etc/ceph/cluster1.client.mirror.keyring && sudo chmod 0644 /etc/ceph/cluster1.client.mirror.keyring 2020-04-10T07:03:49.055 INFO:teuthology.orchestra.run.smithi204.stdout:creating /etc/ceph/cluster1.client.mirror.keyring 2020-04-10T07:03:49.067 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:03:49.116 INFO:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.0 /etc/ceph/cluster1.client.mirror.0.keyring && sudo chmod 0644 /etc/ceph/cluster1.client.mirror.0.keyring 2020-04-10T07:03:49.192 INFO:teuthology.orchestra.run.smithi204.stdout:creating /etc/ceph/cluster1.client.mirror.0.keyring 2020-04-10T07:03:49.204 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:03:49.256 INFO:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.1 /etc/ceph/cluster1.client.mirror.1.keyring && sudo chmod 0644 /etc/ceph/cluster1.client.mirror.1.keyring 2020-04-10T07:03:49.331 INFO:teuthology.orchestra.run.smithi204.stdout:creating /etc/ceph/cluster1.client.mirror.1.keyring 2020-04-10T07:03:49.339 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:03:49.389 INFO:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.2 /etc/ceph/cluster1.client.mirror.2.keyring && sudo chmod 0644 /etc/ceph/cluster1.client.mirror.2.keyring 2020-04-10T07:03:49.463 INFO:teuthology.orchestra.run.smithi204.stdout:creating /etc/ceph/cluster1.client.mirror.2.keyring 2020-04-10T07:03:49.474 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:03:49.522 INFO:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.3 /etc/ceph/cluster1.client.mirror.3.keyring && sudo chmod 0644 /etc/ceph/cluster1.client.mirror.3.keyring 2020-04-10T07:03:49.598 INFO:teuthology.orchestra.run.smithi204.stdout:creating /etc/ceph/cluster1.client.mirror.3.keyring 2020-04-10T07:03:49.609 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:03:49.656 INFO:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.4 /etc/ceph/cluster1.client.mirror.4.keyring && sudo chmod 0644 /etc/ceph/cluster1.client.mirror.4.keyring 2020-04-10T07:03:49.733 INFO:teuthology.orchestra.run.smithi204.stdout:creating /etc/ceph/cluster1.client.mirror.4.keyring 2020-04-10T07:03:49.744 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:03:49.792 INFO:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.5 /etc/ceph/cluster1.client.mirror.5.keyring && sudo chmod 0644 /etc/ceph/cluster1.client.mirror.5.keyring 2020-04-10T07:03:49.867 INFO:teuthology.orchestra.run.smithi204.stdout:creating /etc/ceph/cluster1.client.mirror.5.keyring 2020-04-10T07:03:49.879 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:03:49.928 INFO:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.6 /etc/ceph/cluster1.client.mirror.6.keyring && sudo chmod 0644 /etc/ceph/cluster1.client.mirror.6.keyring 2020-04-10T07:03:50.002 INFO:teuthology.orchestra.run.smithi204.stdout:creating /etc/ceph/cluster1.client.mirror.6.keyring 2020-04-10T07:03:50.015 INFO:tasks.ceph:Running mkfs on osd nodes... 2020-04-10T07:03:50.016 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi101.front.sepia.ceph.com'): {'cluster1.osd.2': '/dev/vg_nvme/lv_2', 'cluster1.osd.1': '/dev/vg_nvme/lv_3', 'cluster1.osd.0': '/dev/vg_nvme/lv_4'}} 2020-04-10T07:03:50.016 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:03:50.021 INFO:teuthology.orchestra.run.smithi101:> sudo mkdir -p /var/lib/ceph/osd/cluster1-0 2020-04-10T07:03:50.081 INFO:tasks.ceph:roles_to_devs: {'cluster1.osd.2': '/dev/vg_nvme/lv_2', 'cluster1.osd.1': '/dev/vg_nvme/lv_3', 'cluster1.osd.0': '/dev/vg_nvme/lv_4'} 2020-04-10T07:03:50.081 INFO:tasks.ceph:role: cluster1.osd.0 2020-04-10T07:03:50.081 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi101.front.sepia.ceph.com 2020-04-10T07:03:50.082 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:03:50.130 INFO:teuthology.orchestra.run.smithi101:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2020-04-10T07:03:50.653 INFO:teuthology.orchestra.run.smithi101.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2020-04-10T07:03:50.653 INFO:teuthology.orchestra.run.smithi101.stdout: = sectsz=512 attr=2, projid32bit=1 2020-04-10T07:03:50.654 INFO:teuthology.orchestra.run.smithi101.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2020-04-10T07:03:50.654 INFO:teuthology.orchestra.run.smithi101.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-04-10T07:03:50.654 INFO:teuthology.orchestra.run.smithi101.stdout: = sunit=0 swidth=0 blks 2020-04-10T07:03:50.654 INFO:teuthology.orchestra.run.smithi101.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2020-04-10T07:03:50.654 INFO:teuthology.orchestra.run.smithi101.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-04-10T07:03:50.654 INFO:teuthology.orchestra.run.smithi101.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-04-10T07:03:50.654 INFO:teuthology.orchestra.run.smithi101.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-04-10T07:03:50.655 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi101.front.sepia.ceph.com -o noatime 2020-04-10T07:03:50.656 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:03:50.661 INFO:teuthology.orchestra.run.smithi101:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/cluster1-0 2020-04-10T07:03:50.775 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:03:50.780 INFO:teuthology.orchestra.run.smithi101:> sudo /sbin/restorecon /var/lib/ceph/osd/cluster1-0 2020-04-10T07:03:50.836 INFO:teuthology.orchestra.run.smithi101.stderr:sudo: /sbin/restorecon: command not found 2020-04-10T07:03:50.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-04-10T07:03:50.837 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:03:50.882 INFO:teuthology.orchestra.run.smithi101:> sudo mkdir -p /var/lib/ceph/osd/cluster1-1 2020-04-10T07:03:50.940 INFO:tasks.ceph:roles_to_devs: {'cluster1.osd.2': '/dev/vg_nvme/lv_2', 'cluster1.osd.1': '/dev/vg_nvme/lv_3', 'cluster1.osd.0': '/dev/vg_nvme/lv_4'} 2020-04-10T07:03:50.940 INFO:tasks.ceph:role: cluster1.osd.1 2020-04-10T07:03:50.940 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi101.front.sepia.ceph.com 2020-04-10T07:03:50.940 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:03:50.986 INFO:teuthology.orchestra.run.smithi101:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2020-04-10T07:03:51.462 INFO:teuthology.orchestra.run.smithi101.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2020-04-10T07:03:51.462 INFO:teuthology.orchestra.run.smithi101.stdout: = sectsz=512 attr=2, projid32bit=1 2020-04-10T07:03:51.462 INFO:teuthology.orchestra.run.smithi101.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2020-04-10T07:03:51.462 INFO:teuthology.orchestra.run.smithi101.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-04-10T07:03:51.462 INFO:teuthology.orchestra.run.smithi101.stdout: = sunit=0 swidth=0 blks 2020-04-10T07:03:51.462 INFO:teuthology.orchestra.run.smithi101.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2020-04-10T07:03:51.463 INFO:teuthology.orchestra.run.smithi101.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-04-10T07:03:51.463 INFO:teuthology.orchestra.run.smithi101.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-04-10T07:03:51.463 INFO:teuthology.orchestra.run.smithi101.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-04-10T07:03:51.464 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi101.front.sepia.ceph.com -o noatime 2020-04-10T07:03:51.464 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:03:51.469 INFO:teuthology.orchestra.run.smithi101:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/cluster1-1 2020-04-10T07:03:51.554 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:03:51.602 INFO:teuthology.orchestra.run.smithi101:> sudo /sbin/restorecon /var/lib/ceph/osd/cluster1-1 2020-04-10T07:03:51.656 INFO:teuthology.orchestra.run.smithi101.stderr:sudo: /sbin/restorecon: command not found 2020-04-10T07:03:51.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-04-10T07:03:51.657 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:03:51.702 INFO:teuthology.orchestra.run.smithi101:> sudo mkdir -p /var/lib/ceph/osd/cluster1-2 2020-04-10T07:03:51.758 INFO:tasks.ceph:roles_to_devs: {'cluster1.osd.2': '/dev/vg_nvme/lv_2', 'cluster1.osd.1': '/dev/vg_nvme/lv_3', 'cluster1.osd.0': '/dev/vg_nvme/lv_4'} 2020-04-10T07:03:51.758 INFO:tasks.ceph:role: cluster1.osd.2 2020-04-10T07:03:51.758 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi101.front.sepia.ceph.com 2020-04-10T07:03:51.759 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:03:51.806 INFO:teuthology.orchestra.run.smithi101:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2020-04-10T07:03:52.338 INFO:teuthology.orchestra.run.smithi101.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2020-04-10T07:03:52.338 INFO:teuthology.orchestra.run.smithi101.stdout: = sectsz=512 attr=2, projid32bit=1 2020-04-10T07:03:52.338 INFO:teuthology.orchestra.run.smithi101.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2020-04-10T07:03:52.338 INFO:teuthology.orchestra.run.smithi101.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-04-10T07:03:52.339 INFO:teuthology.orchestra.run.smithi101.stdout: = sunit=0 swidth=0 blks 2020-04-10T07:03:52.339 INFO:teuthology.orchestra.run.smithi101.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2020-04-10T07:03:52.339 INFO:teuthology.orchestra.run.smithi101.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-04-10T07:03:52.339 INFO:teuthology.orchestra.run.smithi101.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-04-10T07:03:52.339 INFO:teuthology.orchestra.run.smithi101.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-04-10T07:03:52.340 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi101.front.sepia.ceph.com -o noatime 2020-04-10T07:03:52.340 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:03:52.346 INFO:teuthology.orchestra.run.smithi101:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/cluster1-2 2020-04-10T07:03:52.457 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:03:52.463 INFO:teuthology.orchestra.run.smithi101:> sudo /sbin/restorecon /var/lib/ceph/osd/cluster1-2 2020-04-10T07:03:52.516 INFO:teuthology.orchestra.run.smithi101.stderr:sudo: /sbin/restorecon: command not found 2020-04-10T07:03:52.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-04-10T07:03:52.517 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:03:52.566 INFO:teuthology.orchestra.run.smithi101:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster cluster1 --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/cluster1.monmap 2020-04-10T07:03:52.672 INFO:teuthology.orchestra.run.smithi101.stderr:2020-04-10T07:03:52.670+0000 7f45aab3bd80 -1 auth: error reading file: /var/lib/ceph/osd/cluster1-0/keyring: can't open /var/lib/ceph/osd/cluster1-0/keyring: (2) No such file or directory 2020-04-10T07:03:52.675 INFO:teuthology.orchestra.run.smithi101.stderr:2020-04-10T07:03:52.674+0000 7f45aab3bd80 -1 created new key in keyring /var/lib/ceph/osd/cluster1-0/keyring 2020-04-10T07:03:52.675 INFO:teuthology.orchestra.run.smithi101.stderr:2020-04-10T07:03:52.674+0000 7f45aab3bd80 -1 bluestore(/var/lib/ceph/osd/cluster1-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster1-0/block: (2) No such file or directory 2020-04-10T07:03:52.675 INFO:teuthology.orchestra.run.smithi101.stderr:2020-04-10T07:03:52.674+0000 7f45aab3bd80 -1 bluestore(/var/lib/ceph/osd/cluster1-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster1-0/block: (2) No such file or directory 2020-04-10T07:03:52.675 INFO:teuthology.orchestra.run.smithi101.stderr:2020-04-10T07:03:52.674+0000 7f45aab3bd80 -1 bluestore(/var/lib/ceph/osd/cluster1-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster1-0/block: (2) No such file or directory 2020-04-10T07:03:52.680 INFO:teuthology.orchestra.run.smithi101.stderr:2020-04-10T07:03:52.678+0000 7f45aab3bd80 -1 bluestore(/var/lib/ceph/osd/cluster1-0) _read_fsid unparsable uuid 2020-04-10T07:03:55.124 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:03:55.130 INFO:teuthology.orchestra.run.smithi101:> sudo chown -R ceph:ceph /var/lib/ceph/osd/cluster1-0 2020-04-10T07:03:55.188 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:03:55.234 INFO:teuthology.orchestra.run.smithi101:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster cluster1 --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/cluster1.monmap 2020-04-10T07:03:55.315 INFO:teuthology.orchestra.run.smithi101.stderr:2020-04-10T07:03:55.314+0000 7f127f7f9d80 -1 auth: error reading file: /var/lib/ceph/osd/cluster1-1/keyring: can't open /var/lib/ceph/osd/cluster1-1/keyring: (2) No such file or directory 2020-04-10T07:03:55.315 INFO:teuthology.orchestra.run.smithi101.stderr:2020-04-10T07:03:55.314+0000 7f127f7f9d80 -1 created new key in keyring /var/lib/ceph/osd/cluster1-1/keyring 2020-04-10T07:03:55.315 INFO:teuthology.orchestra.run.smithi101.stderr:2020-04-10T07:03:55.314+0000 7f127f7f9d80 -1 bluestore(/var/lib/ceph/osd/cluster1-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster1-1/block: (2) No such file or directory 2020-04-10T07:03:55.315 INFO:teuthology.orchestra.run.smithi101.stderr:2020-04-10T07:03:55.314+0000 7f127f7f9d80 -1 bluestore(/var/lib/ceph/osd/cluster1-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster1-1/block: (2) No such file or directory 2020-04-10T07:03:55.315 INFO:teuthology.orchestra.run.smithi101.stderr:2020-04-10T07:03:55.314+0000 7f127f7f9d80 -1 bluestore(/var/lib/ceph/osd/cluster1-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster1-1/block: (2) No such file or directory 2020-04-10T07:03:55.316 INFO:teuthology.orchestra.run.smithi101.stderr:2020-04-10T07:03:55.314+0000 7f127f7f9d80 -1 bluestore(/var/lib/ceph/osd/cluster1-1) _read_fsid unparsable uuid 2020-04-10T07:03:57.800 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:03:57.846 INFO:teuthology.orchestra.run.smithi101:> sudo chown -R ceph:ceph /var/lib/ceph/osd/cluster1-1 2020-04-10T07:03:57.904 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:03:57.950 INFO:teuthology.orchestra.run.smithi101:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster cluster1 --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/cluster1.monmap 2020-04-10T07:03:58.031 INFO:teuthology.orchestra.run.smithi101.stderr:2020-04-10T07:03:58.030+0000 7fc8ec248d80 -1 auth: error reading file: /var/lib/ceph/osd/cluster1-2/keyring: can't open /var/lib/ceph/osd/cluster1-2/keyring: (2) No such file or directory 2020-04-10T07:03:58.031 INFO:teuthology.orchestra.run.smithi101.stderr:2020-04-10T07:03:58.030+0000 7fc8ec248d80 -1 created new key in keyring /var/lib/ceph/osd/cluster1-2/keyring 2020-04-10T07:03:58.031 INFO:teuthology.orchestra.run.smithi101.stderr:2020-04-10T07:03:58.030+0000 7fc8ec248d80 -1 bluestore(/var/lib/ceph/osd/cluster1-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster1-2/block: (2) No such file or directory 2020-04-10T07:03:58.031 INFO:teuthology.orchestra.run.smithi101.stderr:2020-04-10T07:03:58.030+0000 7fc8ec248d80 -1 bluestore(/var/lib/ceph/osd/cluster1-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster1-2/block: (2) No such file or directory 2020-04-10T07:03:58.031 INFO:teuthology.orchestra.run.smithi101.stderr:2020-04-10T07:03:58.030+0000 7fc8ec248d80 -1 bluestore(/var/lib/ceph/osd/cluster1-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster1-2/block: (2) No such file or directory 2020-04-10T07:03:58.032 INFO:teuthology.orchestra.run.smithi101.stderr:2020-04-10T07:03:58.030+0000 7fc8ec248d80 -1 bluestore(/var/lib/ceph/osd/cluster1-2) _read_fsid unparsable uuid 2020-04-10T07:04:00.487 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:00.534 INFO:teuthology.orchestra.run.smithi101:> sudo chown -R ceph:ceph /var/lib/ceph/osd/cluster1-2 2020-04-10T07:04:00.592 INFO:tasks.ceph:Reading keys from all nodes... 2020-04-10T07:04:00.592 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:00.638 INFO:teuthology.orchestra.run.smithi101:> mktemp 2020-04-10T07:04:00.686 INFO:teuthology.orchestra.run.smithi101.stdout:/tmp/tmp.j4EFYGgh2O 2020-04-10T07:04:00.687 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:00.737 INFO:teuthology.orchestra.run.smithi101:> sudo cp /var/lib/ceph/mgr/cluster1-x/keyring /tmp/tmp.j4EFYGgh2O 2020-04-10T07:04:00.799 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:00.846 INFO:teuthology.orchestra.run.smithi101:> sudo chmod 0666 /tmp/tmp.j4EFYGgh2O 2020-04-10T07:04:00.953 DEBUG:teuthology.orchestra.remote:smithi101:/tmp/tmp.j4EFYGgh2O is 56B 2020-04-10T07:04:00.965 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:00.970 INFO:teuthology.orchestra.run.smithi101:> rm -fr /tmp/tmp.j4EFYGgh2O 2020-04-10T07:04:01.019 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:01.066 INFO:teuthology.orchestra.run.smithi101:> mktemp 2020-04-10T07:04:01.115 INFO:teuthology.orchestra.run.smithi101.stdout:/tmp/tmp.1K7hZ6pZ5t 2020-04-10T07:04:01.115 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:01.162 INFO:teuthology.orchestra.run.smithi101:> sudo cp /var/lib/ceph/osd/cluster1-0/keyring /tmp/tmp.1K7hZ6pZ5t 2020-04-10T07:04:01.220 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:01.267 INFO:teuthology.orchestra.run.smithi101:> sudo chmod 0666 /tmp/tmp.1K7hZ6pZ5t 2020-04-10T07:04:01.378 DEBUG:teuthology.orchestra.remote:smithi101:/tmp/tmp.1K7hZ6pZ5t is 56B 2020-04-10T07:04:01.389 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:01.394 INFO:teuthology.orchestra.run.smithi101:> rm -fr /tmp/tmp.1K7hZ6pZ5t 2020-04-10T07:04:01.443 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:01.490 INFO:teuthology.orchestra.run.smithi101:> mktemp 2020-04-10T07:04:01.539 INFO:teuthology.orchestra.run.smithi101.stdout:/tmp/tmp.RitxaIAxqT 2020-04-10T07:04:01.539 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:01.586 INFO:teuthology.orchestra.run.smithi101:> sudo cp /var/lib/ceph/osd/cluster1-1/keyring /tmp/tmp.RitxaIAxqT 2020-04-10T07:04:01.644 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:01.690 INFO:teuthology.orchestra.run.smithi101:> sudo chmod 0666 /tmp/tmp.RitxaIAxqT 2020-04-10T07:04:01.799 DEBUG:teuthology.orchestra.remote:smithi101:/tmp/tmp.RitxaIAxqT is 56B 2020-04-10T07:04:01.850 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:01.855 INFO:teuthology.orchestra.run.smithi101:> rm -fr /tmp/tmp.RitxaIAxqT 2020-04-10T07:04:01.903 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:01.950 INFO:teuthology.orchestra.run.smithi101:> mktemp 2020-04-10T07:04:01.998 INFO:teuthology.orchestra.run.smithi101.stdout:/tmp/tmp.vd0BU4m6kv 2020-04-10T07:04:01.999 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:02.046 INFO:teuthology.orchestra.run.smithi101:> sudo cp /var/lib/ceph/osd/cluster1-2/keyring /tmp/tmp.vd0BU4m6kv 2020-04-10T07:04:02.104 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:02.150 INFO:teuthology.orchestra.run.smithi101:> sudo chmod 0666 /tmp/tmp.vd0BU4m6kv 2020-04-10T07:04:02.269 DEBUG:teuthology.orchestra.remote:smithi101:/tmp/tmp.vd0BU4m6kv is 56B 2020-04-10T07:04:02.279 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:02.283 INFO:teuthology.orchestra.run.smithi101:> rm -fr /tmp/tmp.vd0BU4m6kv 2020-04-10T07:04:02.381 DEBUG:teuthology.orchestra.remote:smithi101:/etc/ceph/cluster1.client.0.keyring is 59B 2020-04-10T07:04:02.400 DEBUG:teuthology.orchestra.remote:smithi204:/etc/ceph/cluster1.client.mirror.keyring is 64B 2020-04-10T07:04:02.418 DEBUG:teuthology.orchestra.remote:smithi204:/etc/ceph/cluster1.client.mirror.0.keyring is 66B 2020-04-10T07:04:02.437 DEBUG:teuthology.orchestra.remote:smithi204:/etc/ceph/cluster1.client.mirror.1.keyring is 66B 2020-04-10T07:04:02.455 DEBUG:teuthology.orchestra.remote:smithi204:/etc/ceph/cluster1.client.mirror.2.keyring is 66B 2020-04-10T07:04:02.475 DEBUG:teuthology.orchestra.remote:smithi204:/etc/ceph/cluster1.client.mirror.3.keyring is 66B 2020-04-10T07:04:02.493 DEBUG:teuthology.orchestra.remote:smithi204:/etc/ceph/cluster1.client.mirror.4.keyring is 66B 2020-04-10T07:04:02.557 DEBUG:teuthology.orchestra.remote:smithi204:/etc/ceph/cluster1.client.mirror.5.keyring is 66B 2020-04-10T07:04:02.575 DEBUG:teuthology.orchestra.remote:smithi204:/etc/ceph/cluster1.client.mirror.6.keyring is 66B 2020-04-10T07:04:02.585 INFO:tasks.ceph:Adding keys to all mons... 2020-04-10T07:04:02.586 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:02.591 INFO:teuthology.orchestra.run.smithi101:> sudo tee -a /etc/ceph/cluster1.keyring 2020-04-10T07:04:02.646 INFO:teuthology.orchestra.run.smithi101.stdout:[mgr.x] 2020-04-10T07:04:02.646 INFO:teuthology.orchestra.run.smithi101.stdout: key = AQBUGpBe9LgMMhAAnWsP429Y7//YmUstn7wZBg== 2020-04-10T07:04:02.646 INFO:teuthology.orchestra.run.smithi101.stdout:[osd.0] 2020-04-10T07:04:02.646 INFO:teuthology.orchestra.run.smithi101.stdout: key = AQBYGpBe3+AuKBAAuY53Z0yo4YJ3EBfWJNeNtA== 2020-04-10T07:04:02.647 INFO:teuthology.orchestra.run.smithi101.stdout:[osd.1] 2020-04-10T07:04:02.647 INFO:teuthology.orchestra.run.smithi101.stdout: key = AQBbGpBeJRHcEhAAKmhx4NOjtgscEt07ix08kw== 2020-04-10T07:04:02.647 INFO:teuthology.orchestra.run.smithi101.stdout:[osd.2] 2020-04-10T07:04:02.647 INFO:teuthology.orchestra.run.smithi101.stdout: key = AQBeGpBeXf7uARAAME7x+9duMt5tYVWW+Vpk+Q== 2020-04-10T07:04:02.647 INFO:teuthology.orchestra.run.smithi101.stdout:[client.0] 2020-04-10T07:04:02.647 INFO:teuthology.orchestra.run.smithi101.stdout: key = AQBUGpBeyPt3ORAAIhPMLS1SkyBfQCZaPVaK1A== 2020-04-10T07:04:02.647 INFO:teuthology.orchestra.run.smithi101.stdout:[client.mirror] 2020-04-10T07:04:02.648 INFO:teuthology.orchestra.run.smithi101.stdout: key = AQBVGpBe9VFVAxAANg2PFrNAr0TsHfmC0ps35Q== 2020-04-10T07:04:02.648 INFO:teuthology.orchestra.run.smithi101.stdout:[client.mirror.0] 2020-04-10T07:04:02.648 INFO:teuthology.orchestra.run.smithi101.stdout: key = AQBVGpBefd94CxAATZdj8JoQmh+bdrE2m3lfhQ== 2020-04-10T07:04:02.648 INFO:teuthology.orchestra.run.smithi101.stdout:[client.mirror.1] 2020-04-10T07:04:02.648 INFO:teuthology.orchestra.run.smithi101.stdout: key = AQBVGpBexVnIExAAmdF6yuZbOfF3euCsJHA4nQ== 2020-04-10T07:04:02.648 INFO:teuthology.orchestra.run.smithi101.stdout:[client.mirror.2] 2020-04-10T07:04:02.648 INFO:teuthology.orchestra.run.smithi101.stdout: key = AQBVGpBeT/mgGxAAtgI4g1ZT3dtxWCgsG8kNYw== 2020-04-10T07:04:02.649 INFO:teuthology.orchestra.run.smithi101.stdout:[client.mirror.3] 2020-04-10T07:04:02.649 INFO:teuthology.orchestra.run.smithi101.stdout: key = AQBVGpBegwGAIxAAXnG5jyJY6S3MZQ5LjQU7ZQ== 2020-04-10T07:04:02.649 INFO:teuthology.orchestra.run.smithi101.stdout:[client.mirror.4] 2020-04-10T07:04:02.649 INFO:teuthology.orchestra.run.smithi101.stdout: key = AQBVGpBevz6iKxAA24OjgUy684yNGWYvLQbB8A== 2020-04-10T07:04:02.649 INFO:teuthology.orchestra.run.smithi101.stdout:[client.mirror.5] 2020-04-10T07:04:02.649 INFO:teuthology.orchestra.run.smithi101.stdout: key = AQBVGpBeAtW2MxAAhzwUiFxtSBXpG4t6W8/3eA== 2020-04-10T07:04:02.649 INFO:teuthology.orchestra.run.smithi101.stdout:[client.mirror.6] 2020-04-10T07:04:02.650 INFO:teuthology.orchestra.run.smithi101.stdout: key = AQBWGpBePUo0ABAA1Vt3A7I1zmXk2ftregXuIQ== 2020-04-10T07:04:02.650 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:02.698 INFO:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=mgr.x --cap mds 'allow *' --cap osd 'allow *' --cap mon 'allow profile mgr' 2020-04-10T07:04:02.785 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:02.834 INFO:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=osd.0 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2020-04-10T07:04:02.923 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:02.970 INFO:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=osd.1 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2020-04-10T07:04:03.055 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:03.102 INFO:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=osd.2 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2020-04-10T07:04:03.187 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:03.234 INFO:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=client.0 --cap mds allow --cap osd 'allow rwx' --cap mgr 'allow r' --cap mon 'allow rw' 2020-04-10T07:04:03.321 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:03.366 INFO:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=client.mirror --cap mds allow --cap osd 'allow rwx' --cap mgr 'allow r' --cap mon 'allow rw' 2020-04-10T07:04:03.453 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:03.502 INFO:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=client.mirror.0 --cap mds allow --cap osd 'allow rwx' --cap mgr 'allow r' --cap mon 'allow rw' 2020-04-10T07:04:03.587 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:03.634 INFO:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=client.mirror.1 --cap mds allow --cap osd 'allow rwx' --cap mgr 'allow r' --cap mon 'allow rw' 2020-04-10T07:04:03.721 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:03.766 INFO:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=client.mirror.2 --cap mds allow --cap osd 'allow rwx' --cap mgr 'allow r' --cap mon 'allow rw' 2020-04-10T07:04:03.862 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:03.920 INFO:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=client.mirror.3 --cap mds allow --cap osd 'allow rwx' --cap mgr 'allow r' --cap mon 'allow rw' 2020-04-10T07:04:04.020 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:04.071 INFO:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=client.mirror.4 --cap mds allow --cap osd 'allow rwx' --cap mgr 'allow r' --cap mon 'allow rw' 2020-04-10T07:04:04.159 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:04.164 INFO:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=client.mirror.5 --cap mds allow --cap osd 'allow rwx' --cap mgr 'allow r' --cap mon 'allow rw' 2020-04-10T07:04:04.251 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:04.256 INFO:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=client.mirror.6 --cap mds allow --cap osd 'allow rwx' --cap mgr 'allow r' --cap mon 'allow rw' 2020-04-10T07:04:04.342 INFO:tasks.ceph:Running mkfs on mon nodes... 2020-04-10T07:04:04.343 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:04.353 INFO:teuthology.orchestra.run.smithi101:> sudo mkdir -p /var/lib/ceph/mon/cluster1-a 2020-04-10T07:04:04.418 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:04.471 INFO:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster cluster1 --mkfs -i a --monmap /home/ubuntu/cephtest/cluster1.monmap --keyring /etc/ceph/cluster1.keyring 2020-04-10T07:04:04.587 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:04.592 INFO:teuthology.orchestra.run.smithi101:> sudo chown -R ceph:ceph /var/lib/ceph/mon/cluster1-a 2020-04-10T07:04:04.648 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:04.694 INFO:teuthology.orchestra.run.smithi101:> rm -- /home/ubuntu/cephtest/cluster1.monmap 2020-04-10T07:04:04.742 INFO:tasks.ceph:Starting mon daemons in cluster cluster1... 2020-04-10T07:04:04.743 INFO:tasks.ceph.cluster1.mon.a:Restarting daemon 2020-04-10T07:04:04.743 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:04.790 INFO:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster cluster1 -i a 2020-04-10T07:04:04.834 INFO:tasks.ceph.cluster1.mon.a:Started 2020-04-10T07:04:04.835 INFO:tasks.ceph:Starting mgr daemons in cluster cluster1... 2020-04-10T07:04:04.835 INFO:tasks.ceph.cluster1.mgr.x:Restarting daemon 2020-04-10T07:04:04.835 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:04.840 INFO:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster cluster1 -i x 2020-04-10T07:04:04.886 INFO:tasks.ceph.cluster1.mgr.x:Started 2020-04-10T07:04:04.887 INFO:tasks.ceph:Setting crush tunables to default 2020-04-10T07:04:04.887 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:04.892 INFO:teuthology.orchestra.run.smithi101:> sudo ceph --cluster cluster1 osd crush tunables default 2020-04-10T07:04:05.245 INFO:teuthology.orchestra.run.smithi101.stderr:adjusted tunables profile to default 2020-04-10T07:04:05.259 INFO:tasks.ceph:Starting osd daemons in cluster cluster1... 2020-04-10T07:04:05.259 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:05.263 INFO:teuthology.orchestra.run.smithi101:> mktemp 2020-04-10T07:04:05.315 INFO:teuthology.orchestra.run.smithi101.stdout:/tmp/tmp.4cKCOzduqw 2020-04-10T07:04:05.315 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:05.361 INFO:teuthology.orchestra.run.smithi101:> sudo cp /var/lib/ceph/osd/cluster1-0/fsid /tmp/tmp.4cKCOzduqw 2020-04-10T07:04:05.421 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:05.469 INFO:teuthology.orchestra.run.smithi101:> sudo chmod 0666 /tmp/tmp.4cKCOzduqw 2020-04-10T07:04:05.580 DEBUG:teuthology.orchestra.remote:smithi101:/tmp/tmp.4cKCOzduqw is 37B 2020-04-10T07:04:05.629 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:05.634 INFO:teuthology.orchestra.run.smithi101:> rm -fr /tmp/tmp.4cKCOzduqw 2020-04-10T07:04:05.683 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:05.729 INFO:teuthology.orchestra.run.smithi101:> mktemp 2020-04-10T07:04:05.792 INFO:teuthology.orchestra.run.smithi101.stdout:/tmp/tmp.VwgkRsHLxu 2020-04-10T07:04:05.792 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:05.837 INFO:teuthology.orchestra.run.smithi101:> sudo cp /var/lib/ceph/osd/cluster1-1/fsid /tmp/tmp.VwgkRsHLxu 2020-04-10T07:04:05.894 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:05.941 INFO:teuthology.orchestra.run.smithi101:> sudo chmod 0666 /tmp/tmp.VwgkRsHLxu 2020-04-10T07:04:06.048 DEBUG:teuthology.orchestra.remote:smithi101:/tmp/tmp.VwgkRsHLxu is 37B 2020-04-10T07:04:06.097 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:06.102 INFO:teuthology.orchestra.run.smithi101:> rm -fr /tmp/tmp.VwgkRsHLxu 2020-04-10T07:04:06.150 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:06.197 INFO:teuthology.orchestra.run.smithi101:> mktemp 2020-04-10T07:04:06.246 INFO:teuthology.orchestra.run.smithi101.stdout:/tmp/tmp.2XeUZPSJ3R 2020-04-10T07:04:06.246 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:06.293 INFO:teuthology.orchestra.run.smithi101:> sudo cp /var/lib/ceph/osd/cluster1-2/fsid /tmp/tmp.2XeUZPSJ3R 2020-04-10T07:04:06.351 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:06.397 INFO:teuthology.orchestra.run.smithi101:> sudo chmod 0666 /tmp/tmp.2XeUZPSJ3R 2020-04-10T07:04:06.504 DEBUG:teuthology.orchestra.remote:smithi101:/tmp/tmp.2XeUZPSJ3R is 37B 2020-04-10T07:04:06.514 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:06.522 INFO:teuthology.orchestra.run.smithi101:> rm -fr /tmp/tmp.2XeUZPSJ3R 2020-04-10T07:04:06.576 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:06.622 INFO:teuthology.orchestra.run.smithi101:> sudo ceph --cluster cluster1 osd new 9275f125-89e9-48a3-995e-1361b8ebef17 0 2020-04-10T07:04:06.875 INFO:teuthology.orchestra.run.smithi101.stdout:0 2020-04-10T07:04:06.890 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:06.894 INFO:teuthology.orchestra.run.smithi101:> sudo ceph --cluster cluster1 osd new 3c4c3880-4851-480f-8ab5-30b08882dc78 1 2020-04-10T07:04:07.141 INFO:teuthology.orchestra.run.smithi101.stdout:1 2020-04-10T07:04:07.155 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:07.159 INFO:teuthology.orchestra.run.smithi101:> sudo ceph --cluster cluster1 osd new ec9433f0-2cca-4030-835c-9c48f3a3ece5 2 2020-04-10T07:04:07.395 INFO:teuthology.orchestra.run.smithi101.stdout:2 2020-04-10T07:04:07.416 INFO:tasks.ceph.cluster1.osd.0:Restarting daemon 2020-04-10T07:04:07.417 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:07.421 INFO:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster cluster1 -i 0 2020-04-10T07:04:07.466 INFO:tasks.ceph.cluster1.osd.0:Started 2020-04-10T07:04:07.467 INFO:tasks.ceph.cluster1.osd.1:Restarting daemon 2020-04-10T07:04:07.467 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:07.472 INFO:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster cluster1 -i 1 2020-04-10T07:04:07.521 INFO:tasks.ceph.cluster1.osd.1:Started 2020-04-10T07:04:07.521 INFO:tasks.ceph.cluster1.osd.2:Restarting daemon 2020-04-10T07:04:07.521 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:07.525 INFO:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster cluster1 -i 2 2020-04-10T07:04:07.570 INFO:tasks.ceph.cluster1.osd.2:Started 2020-04-10T07:04:07.570 INFO:tasks.ceph:Waiting for OSDs to come up 2020-04-10T07:04:07.822 INFO:tasks.ceph.cluster1.osd.0.smithi101.stderr:2020-04-10T07:04:07.822+0000 7fbf68e24d80 -1 Falling back to public interface 2020-04-10T07:04:07.872 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:07.885 INFO:teuthology.orchestra.run.smithi101:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster cluster1 osd dump --format=json 2020-04-10T07:04:07.887 INFO:tasks.ceph.cluster1.osd.1.smithi101.stderr:2020-04-10T07:04:07.886+0000 7f4c923add80 -1 Falling back to public interface 2020-04-10T07:04:08.054 INFO:teuthology.misc.health.smithi101.stdout: 2020-04-10T07:04:08.054 INFO:teuthology.misc.health.smithi101.stdout:{"epoch":5,"fsid":"5d7eff56-55fb-4da2-8b7d-3ec54a26cfc2","created":"2020-04-10T07:04:04.969913+0000","modified":"2020-04-10T07:04:07.394657+0000","last_up_change":"0.000000","last_in_change":"0.000000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[],"osds":[{"osd":0,"uuid":"9275f125-89e9-48a3-995e-1361b8ebef17","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"3c4c3880-4851-480f-8ab5-30b08882dc78","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"ec9433f0-2cca-4030-835c-9c48f3a3ece5","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-04-10T07:04:08.064 DEBUG:teuthology.misc:0 of 3 OSDs are up 2020-04-10T07:04:08.207 INFO:tasks.ceph.cluster1.osd.2.smithi101.stderr:2020-04-10T07:04:08.206+0000 7f81186a3d80 -1 Falling back to public interface 2020-04-10T07:04:08.995 INFO:tasks.ceph.cluster1.osd.0.smithi101.stderr:2020-04-10T07:04:08.994+0000 7fbf68e24d80 -1 osd.0 0 log_to_monitors {default=true} 2020-04-10T07:04:09.075 INFO:tasks.ceph.cluster1.osd.1.smithi101.stderr:2020-04-10T07:04:09.074+0000 7f4c923add80 -1 osd.1 0 log_to_monitors {default=true} 2020-04-10T07:04:09.123 INFO:tasks.ceph.cluster1.osd.2.smithi101.stderr:2020-04-10T07:04:09.122+0000 7f81186a3d80 -1 osd.2 0 log_to_monitors {default=true} 2020-04-10T07:04:10.612 INFO:tasks.ceph.cluster1.osd.0.smithi101.stderr:2020-04-10T07:04:10.610+0000 7fbf52070700 -1 osd.0 0 waiting for initial osdmap 2020-04-10T07:04:10.612 INFO:tasks.ceph.cluster1.osd.1.smithi101.stderr:2020-04-10T07:04:10.610+0000 7f4c7b5f9700 -1 osd.1 0 waiting for initial osdmap 2020-04-10T07:04:10.612 INFO:tasks.ceph.cluster1.osd.2.smithi101.stderr:2020-04-10T07:04:10.610+0000 7f81018ef700 -1 osd.2 0 waiting for initial osdmap 2020-04-10T07:04:14.368 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:14.374 INFO:teuthology.orchestra.run.smithi101:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster cluster1 osd dump --format=json 2020-04-10T07:04:14.671 INFO:teuthology.misc.health.smithi101.stdout: 2020-04-10T07:04:14.672 INFO:teuthology.misc.health.smithi101.stdout:{"epoch":10,"fsid":"5d7eff56-55fb-4da2-8b7d-3ec54a26cfc2","created":"2020-04-10T07:04:04.969913+0000","modified":"2020-04-10T07:04:13.613090+0000","last_up_change":"2020-04-10T07:04:11.602519+0000","last_in_change":"2020-04-10T07:04:11.602519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-04-10T07:04:08.638115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"10","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}}],"osds":[{"osd":0,"uuid":"9275f125-89e9-48a3-995e-1361b8ebef17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6800","nonce":12364},{"type":"v1","addr":"172.21.15.101:6801","nonce":12364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6802","nonce":12364},{"type":"v1","addr":"172.21.15.101:6803","nonce":12364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6806","nonce":12364},{"type":"v1","addr":"172.21.15.101:6807","nonce":12364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6804","nonce":12364},{"type":"v1","addr":"172.21.15.101:6805","nonce":12364}]},"public_addr":"172.21.15.101:6801/12364","cluster_addr":"172.21.15.101:6803/12364","heartbeat_back_addr":"172.21.15.101:6807/12364","heartbeat_front_addr":"172.21.15.101:6805/12364","state":["exists","up"]},{"osd":1,"uuid":"3c4c3880-4851-480f-8ab5-30b08882dc78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":8,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6808","nonce":12383},{"type":"v1","addr":"172.21.15.101:6809","nonce":12383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6810","nonce":12383},{"type":"v1","addr":"172.21.15.101:6811","nonce":12383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6814","nonce":12383},{"type":"v1","addr":"172.21.15.101:6815","nonce":12383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6812","nonce":12383},{"type":"v1","addr":"172.21.15.101:6813","nonce":12383}]},"public_addr":"172.21.15.101:6809/12383","cluster_addr":"172.21.15.101:6811/12383","heartbeat_back_addr":"172.21.15.101:6815/12383","heartbeat_front_addr":"172.21.15.101:6813/12383","state":["exists","up"]},{"osd":2,"uuid":"ec9433f0-2cca-4030-835c-9c48f3a3ece5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6816","nonce":12401},{"type":"v1","addr":"172.21.15.101:6817","nonce":12401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6818","nonce":12401},{"type":"v1","addr":"172.21.15.101:6819","nonce":12401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6822","nonce":12401},{"type":"v1","addr":"172.21.15.101:6823","nonce":12401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6820","nonce":12401},{"type":"v1","addr":"172.21.15.101:6821","nonce":12401}]},"public_addr":"172.21.15.101:6817/12401","cluster_addr":"172.21.15.101:6819/12401","heartbeat_back_addr":"172.21.15.101:6823/12401","heartbeat_front_addr":"172.21.15.101:6821/12401","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-04-10T07:04:10.032973+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-04-10T07:04:10.115173+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-04-10T07:04:10.124851+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-04-10T07:04:14.684 DEBUG:teuthology.misc:3 of 3 OSDs are up 2020-04-10T07:04:14.684 INFO:tasks.ceph:Creating RBD pool 2020-04-10T07:04:14.684 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:14.690 INFO:teuthology.orchestra.run.smithi101:> sudo ceph --cluster cluster1 osd pool create rbd 8 2020-04-10T07:04:15.938 INFO:teuthology.orchestra.run.smithi101.stderr:pool 'rbd' created 2020-04-10T07:04:15.952 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:15.956 INFO:teuthology.orchestra.run.smithi101:> sudo ceph --cluster cluster1 osd pool application enable rbd rbd --yes-i-really-mean-it 2020-04-10T07:04:16.942 INFO:teuthology.orchestra.run.smithi101.stderr:enabled application 'rbd' on pool 'rbd' 2020-04-10T07:04:16.956 INFO:tasks.ceph:Starting mds daemons in cluster cluster1... 2020-04-10T07:04:16.957 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:16.957 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2020-04-10T07:04:16.961 INFO:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 --log-early osd dump --format=json 2020-04-10T07:04:17.265 INFO:teuthology.orchestra.run.smithi101.stdout: 2020-04-10T07:04:17.265 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":12,"fsid":"5d7eff56-55fb-4da2-8b7d-3ec54a26cfc2","created":"2020-04-10T07:04:04.969913+0000","modified":"2020-04-10T07:04:16.931938+0000","last_up_change":"2020-04-10T07:04:11.602519+0000","last_in_change":"2020-04-10T07:04:11.602519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-04-10T07:04:08.638115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"10","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-04-10T07:04:15.037023+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"9275f125-89e9-48a3-995e-1361b8ebef17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6800","nonce":12364},{"type":"v1","addr":"172.21.15.101:6801","nonce":12364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6802","nonce":12364},{"type":"v1","addr":"172.21.15.101:6803","nonce":12364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6806","nonce":12364},{"type":"v1","addr":"172.21.15.101:6807","nonce":12364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6804","nonce":12364},{"type":"v1","addr":"172.21.15.101:6805","nonce":12364}]},"public_addr":"172.21.15.101:6801/12364","cluster_addr":"172.21.15.101:6803/12364","heartbeat_back_addr":"172.21.15.101:6807/12364","heartbeat_front_addr":"172.21.15.101:6805/12364","state":["exists","up"]},{"osd":1,"uuid":"3c4c3880-4851-480f-8ab5-30b08882dc78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6808","nonce":12383},{"type":"v1","addr":"172.21.15.101:6809","nonce":12383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6810","nonce":12383},{"type":"v1","addr":"172.21.15.101:6811","nonce":12383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6814","nonce":12383},{"type":"v1","addr":"172.21.15.101:6815","nonce":12383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6812","nonce":12383},{"type":"v1","addr":"172.21.15.101:6813","nonce":12383}]},"public_addr":"172.21.15.101:6809/12383","cluster_addr":"172.21.15.101:6811/12383","heartbeat_back_addr":"172.21.15.101:6815/12383","heartbeat_front_addr":"172.21.15.101:6813/12383","state":["exists","up"]},{"osd":2,"uuid":"ec9433f0-2cca-4030-835c-9c48f3a3ece5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6816","nonce":12401},{"type":"v1","addr":"172.21.15.101:6817","nonce":12401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6818","nonce":12401},{"type":"v1","addr":"172.21.15.101:6819","nonce":12401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6822","nonce":12401},{"type":"v1","addr":"172.21.15.101:6823","nonce":12401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6820","nonce":12401},{"type":"v1","addr":"172.21.15.101:6821","nonce":12401}]},"public_addr":"172.21.15.101:6817/12401","cluster_addr":"172.21.15.101:6819/12401","heartbeat_back_addr":"172.21.15.101:6823/12401","heartbeat_front_addr":"172.21.15.101:6821/12401","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-04-10T07:04:10.032973+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-04-10T07:04:10.115173+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-04-10T07:04:10.124851+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-04-10T07:04:17.279 INFO:tasks.ceph.ceph_manager.cluster1:[{u'cache_target_full_ratio_micro': 800000, u'fast_read': False, u'stripe_width': 0, u'flags_names': u'hashpspool', u'tier_of': -1, u'hit_set_grade_decay_rate': 0, u'pg_placement_num': 1, u'use_gmt_hitset': True, u'cache_target_dirty_ratio_micro': 400000, u'create_time': u'2020-04-10T07:04:08.638115+0000', u'snap_epoch': 0, u'erasure_code_profile': u'', u'pg_autoscale_mode': u'off', u'snap_seq': 0, u'expected_num_objects': 0, u'size': 2, u'pg_num_pending': 1, u'auid': 0, u'cache_min_flush_age': 0, u'hit_set_period': 0, u'min_read_recency_for_promote': 0, u'target_max_objects': 0, u'pg_placement_num_target': 1, u'pg_num': 1, u'type': 1, u'grade_table': [], u'pool_name': u'device_health_metrics', u'cache_min_evict_age': 0, u'snap_mode': u'selfmanaged', u'pg_num_target': 1, u'cache_mode': u'none', u'min_size': 1, u'cache_target_dirty_high_ratio_micro': 600000, u'object_hash': 2, u'last_pg_merge_meta': {u'ready_epoch': 0, u'source_version': u"0'0", u'source_pgid': u'0.0', u'last_epoch_clean': 0, u'target_version': u"0'0", u'last_epoch_started': 0}, u'write_tier': -1, u'last_force_op_resend_preluminous': u'0', u'pool': 1, u'removed_snaps': u'[]', u'crush_rule': 0, u'tiers': [], u'hit_set_params': {u'type': u'none'}, u'last_force_op_resend': u'0', u'pool_snaps': [], u'quota_max_objects': 0, u'last_force_op_resend_prenautilus': u'0', u'application_metadata': {u'mgr_devicehealth': {}}, u'options': {u'pg_num_min': 1}, u'hit_set_count': 0, u'flags': 1, u'target_max_bytes': 0, u'quota_max_bytes': 0, u'hit_set_search_last_n': 0, u'last_change': u'10', u'min_write_recency_for_promote': 0, u'read_tier': -1}, {u'cache_target_full_ratio_micro': 800000, u'fast_read': False, u'stripe_width': 0, u'flags_names': u'hashpspool,creating', u'tier_of': -1, u'hit_set_grade_decay_rate': 0, u'pg_placement_num': 8, u'use_gmt_hitset': True, u'cache_target_dirty_ratio_micro': 400000, u'create_time': u'2020-04-10T07:04:15.037023+0000', u'snap_epoch': 0, u'erasure_code_profile': u'', u'pg_autoscale_mode': u'off', u'snap_seq': 0, u'expected_num_objects': 0, u'size': 2, u'pg_num_pending': 8, u'auid': 0, u'cache_min_flush_age': 0, u'hit_set_period': 0, u'min_read_recency_for_promote': 0, u'target_max_objects': 0, u'pg_placement_num_target': 8, u'pg_num': 8, u'type': 1, u'grade_table': [], u'pool_name': u'rbd', u'cache_min_evict_age': 0, u'snap_mode': u'selfmanaged', u'pg_num_target': 8, u'cache_mode': u'none', u'min_size': 1, u'cache_target_dirty_high_ratio_micro': 600000, u'object_hash': 2, u'last_pg_merge_meta': {u'ready_epoch': 0, u'source_version': u"0'0", u'source_pgid': u'0.0', u'last_epoch_clean': 0, u'target_version': u"0'0", u'last_epoch_started': 0}, u'write_tier': -1, u'last_force_op_resend_preluminous': u'0', u'pool': 2, u'removed_snaps': u'[]', u'crush_rule': 0, u'tiers': [], u'hit_set_params': {u'type': u'none'}, u'last_force_op_resend': u'0', u'pool_snaps': [], u'quota_max_objects': 0, u'last_force_op_resend_prenautilus': u'0', u'application_metadata': {u'rbd': {}}, u'options': {}, u'hit_set_count': 0, u'flags': 32769, u'target_max_bytes': 0, u'quota_max_bytes': 0, u'hit_set_search_last_n': 0, u'last_change': u'12', u'min_write_recency_for_promote': 0, u'read_tier': -1}] 2020-04-10T07:04:17.279 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:17.289 INFO:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 --log-early osd pool get device_health_metrics pg_num 2020-04-10T07:04:17.599 INFO:teuthology.orchestra.run.smithi101.stdout:pg_num: 1 2020-04-10T07:04:17.612 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:17.618 INFO:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 --log-early osd pool get rbd pg_num 2020-04-10T07:04:17.932 INFO:teuthology.orchestra.run.smithi101.stdout:pg_num: 8 2020-04-10T07:04:17.948 INFO:tasks.ceph:Waiting until cluster1 daemons up and pgs clean... 2020-04-10T07:04:17.948 INFO:tasks.ceph.ceph_manager.cluster1:waiting for mgr available 2020-04-10T07:04:17.948 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:17.952 INFO:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 --log-early mgr dump --format=json 2020-04-10T07:04:18.286 INFO:teuthology.orchestra.run.smithi101.stdout: 2020-04-10T07:04:18.299 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":4,"active_gid":4099,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6824","nonce":12176},{"type":"v1","addr":"172.21.15.101:6825","nonce":12176}]},"active_addr":"172.21.15.101:6825/12176","active_change":"2020-04-10T07:04:08.585544+0000","active_mgr_features":4540138292836696063,"available":true,"standbys":[],"modules":["iostat","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"none","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading remoto library:No module named 'remoto'","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"docker.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_nfs":{"name":"FEATURE_TOGGLE_nfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_cloud","can_run":true,"error_string":"","module_options":{"diskprediction_cert_context":{"name":"diskprediction_cert_context","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_default_authority":{"name":"diskprediction_default_authority","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_password":{"name":"diskprediction_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_port":{"name":"diskprediction_port","type":"str","level":"advanced","flags":0,"default_value":"31400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_retrieve_prediction_interval":{"name":"diskprediction_retrieve_prediction_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_server":{"name":"diskprediction_server","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_ssl_target_name_override":{"name":"diskprediction_ssl_target_name_override","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_metrics_interval":{"name":"diskprediction_upload_metrics_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_smart_interval":{"name":"diskprediction_upload_smart_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_user":{"name":"diskprediction_user","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"None","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator","osd_support","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.101:0","nonce":4014335156}]},{"addrvec":[{"type":"v2","addr":"172.21.15.101:0","nonce":1953950268}]},{"addrvec":[{"type":"v2","addr":"172.21.15.101:0","nonce":1442998994}]}]}} 2020-04-10T07:04:18.303 INFO:tasks.ceph.ceph_manager.cluster1:mgr available! 2020-04-10T07:04:18.303 INFO:tasks.ceph.ceph_manager.cluster1:waiting for all up 2020-04-10T07:04:18.303 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:18.313 INFO:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 --log-early osd dump --format=json 2020-04-10T07:04:18.615 INFO:teuthology.orchestra.run.smithi101.stdout: 2020-04-10T07:04:18.615 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":13,"fsid":"5d7eff56-55fb-4da2-8b7d-3ec54a26cfc2","created":"2020-04-10T07:04:04.969913+0000","modified":"2020-04-10T07:04:17.933398+0000","last_up_change":"2020-04-10T07:04:11.602519+0000","last_in_change":"2020-04-10T07:04:11.602519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-04-10T07:04:08.638115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"10","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-04-10T07:04:15.037023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"9275f125-89e9-48a3-995e-1361b8ebef17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6800","nonce":12364},{"type":"v1","addr":"172.21.15.101:6801","nonce":12364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6802","nonce":12364},{"type":"v1","addr":"172.21.15.101:6803","nonce":12364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6806","nonce":12364},{"type":"v1","addr":"172.21.15.101:6807","nonce":12364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6804","nonce":12364},{"type":"v1","addr":"172.21.15.101:6805","nonce":12364}]},"public_addr":"172.21.15.101:6801/12364","cluster_addr":"172.21.15.101:6803/12364","heartbeat_back_addr":"172.21.15.101:6807/12364","heartbeat_front_addr":"172.21.15.101:6805/12364","state":["exists","up"]},{"osd":1,"uuid":"3c4c3880-4851-480f-8ab5-30b08882dc78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6808","nonce":12383},{"type":"v1","addr":"172.21.15.101:6809","nonce":12383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6810","nonce":12383},{"type":"v1","addr":"172.21.15.101:6811","nonce":12383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6814","nonce":12383},{"type":"v1","addr":"172.21.15.101:6815","nonce":12383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6812","nonce":12383},{"type":"v1","addr":"172.21.15.101:6813","nonce":12383}]},"public_addr":"172.21.15.101:6809/12383","cluster_addr":"172.21.15.101:6811/12383","heartbeat_back_addr":"172.21.15.101:6815/12383","heartbeat_front_addr":"172.21.15.101:6813/12383","state":["exists","up"]},{"osd":2,"uuid":"ec9433f0-2cca-4030-835c-9c48f3a3ece5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6816","nonce":12401},{"type":"v1","addr":"172.21.15.101:6817","nonce":12401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6818","nonce":12401},{"type":"v1","addr":"172.21.15.101:6819","nonce":12401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6822","nonce":12401},{"type":"v1","addr":"172.21.15.101:6823","nonce":12401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6820","nonce":12401},{"type":"v1","addr":"172.21.15.101:6821","nonce":12401}]},"public_addr":"172.21.15.101:6817/12401","cluster_addr":"172.21.15.101:6819/12401","heartbeat_back_addr":"172.21.15.101:6823/12401","heartbeat_front_addr":"172.21.15.101:6821/12401","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-04-10T07:04:10.032973+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-04-10T07:04:10.115173+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-04-10T07:04:10.124851+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-04-10T07:04:18.632 INFO:tasks.ceph.ceph_manager.cluster1:all up! 2020-04-10T07:04:18.632 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:18.638 INFO:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 --log-early osd dump --format=json 2020-04-10T07:04:18.947 INFO:teuthology.orchestra.run.smithi101.stdout: 2020-04-10T07:04:18.947 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":13,"fsid":"5d7eff56-55fb-4da2-8b7d-3ec54a26cfc2","created":"2020-04-10T07:04:04.969913+0000","modified":"2020-04-10T07:04:17.933398+0000","last_up_change":"2020-04-10T07:04:11.602519+0000","last_in_change":"2020-04-10T07:04:11.602519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-04-10T07:04:08.638115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"10","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-04-10T07:04:15.037023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"9275f125-89e9-48a3-995e-1361b8ebef17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6800","nonce":12364},{"type":"v1","addr":"172.21.15.101:6801","nonce":12364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6802","nonce":12364},{"type":"v1","addr":"172.21.15.101:6803","nonce":12364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6806","nonce":12364},{"type":"v1","addr":"172.21.15.101:6807","nonce":12364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6804","nonce":12364},{"type":"v1","addr":"172.21.15.101:6805","nonce":12364}]},"public_addr":"172.21.15.101:6801/12364","cluster_addr":"172.21.15.101:6803/12364","heartbeat_back_addr":"172.21.15.101:6807/12364","heartbeat_front_addr":"172.21.15.101:6805/12364","state":["exists","up"]},{"osd":1,"uuid":"3c4c3880-4851-480f-8ab5-30b08882dc78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6808","nonce":12383},{"type":"v1","addr":"172.21.15.101:6809","nonce":12383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6810","nonce":12383},{"type":"v1","addr":"172.21.15.101:6811","nonce":12383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6814","nonce":12383},{"type":"v1","addr":"172.21.15.101:6815","nonce":12383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6812","nonce":12383},{"type":"v1","addr":"172.21.15.101:6813","nonce":12383}]},"public_addr":"172.21.15.101:6809/12383","cluster_addr":"172.21.15.101:6811/12383","heartbeat_back_addr":"172.21.15.101:6815/12383","heartbeat_front_addr":"172.21.15.101:6813/12383","state":["exists","up"]},{"osd":2,"uuid":"ec9433f0-2cca-4030-835c-9c48f3a3ece5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6816","nonce":12401},{"type":"v1","addr":"172.21.15.101:6817","nonce":12401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6818","nonce":12401},{"type":"v1","addr":"172.21.15.101:6819","nonce":12401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6822","nonce":12401},{"type":"v1","addr":"172.21.15.101:6823","nonce":12401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6820","nonce":12401},{"type":"v1","addr":"172.21.15.101:6821","nonce":12401}]},"public_addr":"172.21.15.101:6817/12401","cluster_addr":"172.21.15.101:6819/12401","heartbeat_back_addr":"172.21.15.101:6823/12401","heartbeat_front_addr":"172.21.15.101:6821/12401","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-04-10T07:04:10.032973+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-04-10T07:04:10.115173+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-04-10T07:04:10.124851+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-04-10T07:04:18.959 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:18.966 INFO:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 --log-early tell osd.0 flush_pg_stats 2020-04-10T07:04:19.165 INFO:teuthology.orchestra.run.smithi101.stdout:34359738371 2020-04-10T07:04:19.165 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:19.169 INFO:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 --log-early tell osd.1 flush_pg_stats 2020-04-10T07:04:19.373 INFO:teuthology.orchestra.run.smithi101.stdout:34359738371 2020-04-10T07:04:19.373 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:19.378 INFO:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 --log-early tell osd.2 flush_pg_stats 2020-04-10T07:04:19.594 INFO:teuthology.orchestra.run.smithi101.stdout:34359738371 2020-04-10T07:04:19.594 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:19.600 INFO:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 --log-early osd last-stat-seq osd.0 2020-04-10T07:04:19.908 INFO:teuthology.orchestra.run.smithi101.stdout:34359738370 2020-04-10T07:04:19.921 INFO:tasks.ceph.ceph_manager.cluster1:need seq 34359738371 got 34359738370 for osd.0 2020-04-10T07:04:20.922 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:20.928 INFO:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 --log-early osd last-stat-seq osd.0 2020-04-10T07:04:21.233 INFO:teuthology.orchestra.run.smithi101.stdout:34359738371 2020-04-10T07:04:21.246 INFO:tasks.ceph.ceph_manager.cluster1:need seq 34359738371 got 34359738371 for osd.0 2020-04-10T07:04:21.247 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:21.252 INFO:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 --log-early osd last-stat-seq osd.1 2020-04-10T07:04:21.551 INFO:teuthology.orchestra.run.smithi101.stdout:34359738371 2020-04-10T07:04:21.570 INFO:tasks.ceph.ceph_manager.cluster1:need seq 34359738371 got 34359738371 for osd.1 2020-04-10T07:04:21.570 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:21.575 INFO:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 --log-early osd last-stat-seq osd.2 2020-04-10T07:04:21.880 INFO:teuthology.orchestra.run.smithi101.stdout:34359738371 2020-04-10T07:04:21.893 INFO:tasks.ceph.ceph_manager.cluster1:need seq 34359738371 got 34359738371 for osd.2 2020-04-10T07:04:21.893 INFO:tasks.ceph.ceph_manager.cluster1:waiting for clean 2020-04-10T07:04:21.893 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:21.898 INFO:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 --log-early pg dump --format=json 2020-04-10T07:04:22.193 INFO:teuthology.orchestra.run.smithi101.stdout: 2020-04-10T07:04:22.195 INFO:teuthology.orchestra.run.smithi101.stderr:dumped all 2020-04-10T07:04:22.205 INFO:teuthology.orchestra.run.smithi101.stdout:{"pg_ready":true,"pg_map":{"version":16,"stamp":"2020-04-10T07:04:20.596333+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":3146072,"kb_used_data":320,"kb_used_omap":0,"kb_used_meta":3145728,"kb_avail":279969448,"statfs":{"total":289910292480,"available":286688714752,"internally_reserved":3221225472,"allocated":327680,"data_stored":93582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":3221225472},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.661734"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":"8","reported_epoch":"12","state":"active+clean","last_fresh":"2020-04-10T07:04:16.950988+0000","last_change":"2020-04-10T07:04:16.950988+0000","last_active":"2020-04-10T07:04:16.950988+0000","last_peered":"2020-04-10T07:04:16.950988+0000","last_clean":"2020-04-10T07:04:16.950988+0000","last_became_active":"2020-04-10T07:04:16.950572+0000","last_became_peered":"2020-04-10T07:04:16.950572+0000","last_unstale":"2020-04-10T07:04:16.950988+0000","last_undegraded":"2020-04-10T07:04:16.950988+0000","last_fullsized":"2020-04-10T07:04:16.950988+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-04-10T07:04:15.930368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-04-10T07:04:15.930368+0000","last_clean_scrub_stamp":"2020-04-10T07:04:15.930368+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"8","reported_epoch":"12","state":"active+clean","last_fresh":"2020-04-10T07:04:16.950009+0000","last_change":"2020-04-10T07:04:16.950009+0000","last_active":"2020-04-10T07:04:16.950009+0000","last_peered":"2020-04-10T07:04:16.950009+0000","last_clean":"2020-04-10T07:04:16.950009+0000","last_became_active":"2020-04-10T07:04:16.949586+0000","last_became_peered":"2020-04-10T07:04:16.949586+0000","last_unstale":"2020-04-10T07:04:16.950009+0000","last_undegraded":"2020-04-10T07:04:16.950009+0000","last_fullsized":"2020-04-10T07:04:16.950009+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-04-10T07:04:15.930368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-04-10T07:04:15.930368+0000","last_clean_scrub_stamp":"2020-04-10T07:04:15.930368+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"8","reported_epoch":"12","state":"active+clean","last_fresh":"2020-04-10T07:04:16.948903+0000","last_change":"2020-04-10T07:04:16.948903+0000","last_active":"2020-04-10T07:04:16.948903+0000","last_peered":"2020-04-10T07:04:16.948903+0000","last_clean":"2020-04-10T07:04:16.948903+0000","last_became_active":"2020-04-10T07:04:16.948453+0000","last_became_peered":"2020-04-10T07:04:16.948453+0000","last_unstale":"2020-04-10T07:04:16.948903+0000","last_undegraded":"2020-04-10T07:04:16.948903+0000","last_fullsized":"2020-04-10T07:04:16.948903+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-04-10T07:04:15.930368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-04-10T07:04:15.930368+0000","last_clean_scrub_stamp":"2020-04-10T07:04:15.930368+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"8","reported_epoch":"12","state":"active+clean","last_fresh":"2020-04-10T07:04:16.945786+0000","last_change":"2020-04-10T07:04:16.945786+0000","last_active":"2020-04-10T07:04:16.945786+0000","last_peered":"2020-04-10T07:04:16.945786+0000","last_clean":"2020-04-10T07:04:16.945786+0000","last_became_active":"2020-04-10T07:04:16.945274+0000","last_became_peered":"2020-04-10T07:04:16.945274+0000","last_unstale":"2020-04-10T07:04:16.945786+0000","last_undegraded":"2020-04-10T07:04:16.945786+0000","last_fullsized":"2020-04-10T07:04:16.945786+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-04-10T07:04:15.930368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-04-10T07:04:15.930368+0000","last_clean_scrub_stamp":"2020-04-10T07:04:15.930368+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"9'1","reported_seq":"12","reported_epoch":"11","state":"active+clean","last_fresh":"2020-04-10T07:04:15.941720+0000","last_change":"2020-04-10T07:04:12.950377+0000","last_active":"2020-04-10T07:04:15.941720+0000","last_peered":"2020-04-10T07:04:15.941720+0000","last_clean":"2020-04-10T07:04:15.941720+0000","last_became_active":"2020-04-10T07:04:12.948567+0000","last_became_peered":"2020-04-10T07:04:12.948567+0000","last_unstale":"2020-04-10T07:04:15.941720+0000","last_undegraded":"2020-04-10T07:04:15.941720+0000","last_fullsized":"2020-04-10T07:04:15.941720+0000","mapping_epoch":8,"log_start":"0'0","ondisk_log_start":"0'0","created":6,"last_epoch_clean":9,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-04-10T07:04:09.589146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-04-10T07:04:09.589146+0000","last_clean_scrub_stamp":"2020-04-10T07:04:09.589146+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"8","reported_epoch":"12","state":"active+clean","last_fresh":"2020-04-10T07:04:16.947141+0000","last_change":"2020-04-10T07:04:16.947141+0000","last_active":"2020-04-10T07:04:16.947141+0000","last_peered":"2020-04-10T07:04:16.947141+0000","last_clean":"2020-04-10T07:04:16.947141+0000","last_became_active":"2020-04-10T07:04:16.946721+0000","last_became_peered":"2020-04-10T07:04:16.946721+0000","last_unstale":"2020-04-10T07:04:16.947141+0000","last_undegraded":"2020-04-10T07:04:16.947141+0000","last_fullsized":"2020-04-10T07:04:16.947141+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-04-10T07:04:15.930368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-04-10T07:04:15.930368+0000","last_clean_scrub_stamp":"2020-04-10T07:04:15.930368+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"8","reported_epoch":"12","state":"active+clean","last_fresh":"2020-04-10T07:04:16.949437+0000","last_change":"2020-04-10T07:04:16.949437+0000","last_active":"2020-04-10T07:04:16.949437+0000","last_peered":"2020-04-10T07:04:16.949437+0000","last_clean":"2020-04-10T07:04:16.949437+0000","last_became_active":"2020-04-10T07:04:16.949010+0000","last_became_peered":"2020-04-10T07:04:16.949010+0000","last_unstale":"2020-04-10T07:04:16.949437+0000","last_undegraded":"2020-04-10T07:04:16.949437+0000","last_fullsized":"2020-04-10T07:04:16.949437+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-04-10T07:04:15.930368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-04-10T07:04:15.930368+0000","last_clean_scrub_stamp":"2020-04-10T07:04:15.930368+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"9","reported_epoch":"12","state":"active+clean","last_fresh":"2020-04-10T07:04:16.951206+0000","last_change":"2020-04-10T07:04:16.950524+0000","last_active":"2020-04-10T07:04:16.951206+0000","last_peered":"2020-04-10T07:04:16.951206+0000","last_clean":"2020-04-10T07:04:16.951206+0000","last_became_active":"2020-04-10T07:04:16.950148+0000","last_became_peered":"2020-04-10T07:04:16.950148+0000","last_unstale":"2020-04-10T07:04:16.951206+0000","last_undegraded":"2020-04-10T07:04:16.951206+0000","last_fullsized":"2020-04-10T07:04:16.951206+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-04-10T07:04:15.930368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-04-10T07:04:15.930368+0000","last_clean_scrub_stamp":"2020-04-10T07:04:15.930368+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"8","reported_epoch":"12","state":"active+clean","last_fresh":"2020-04-10T07:04:16.947801+0000","last_change":"2020-04-10T07:04:16.947801+0000","last_active":"2020-04-10T07:04:16.947801+0000","last_peered":"2020-04-10T07:04:16.947801+0000","last_clean":"2020-04-10T07:04:16.947801+0000","last_became_active":"2020-04-10T07:04:16.947324+0000","last_became_peered":"2020-04-10T07:04:16.947324+0000","last_unstale":"2020-04-10T07:04:16.947801+0000","last_undegraded":"2020-04-10T07:04:16.947801+0000","last_fullsized":"2020-04-10T07:04:16.947801+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-04-10T07:04:15.930368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-04-10T07:04:15.930368+0000","last_clean_scrub_stamp":"2020-04-10T07:04:15.930368+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738371,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048688,"kb_used_data":104,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323152,"statfs":{"total":96636764160,"available":95562907648,"internally_reserved":1073741824,"allocated":106496,"data_stored":30023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.196}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5649999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738371,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048696,"kb_used_data":112,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323144,"statfs":{"total":96636764160,"available":95562899456,"internally_reserved":1073741824,"allocated":114688,"data_stored":33536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0169999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3220000000000001}]}]},{"osd":1,"up_from":8,"seq":34359738371,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048688,"kb_used_data":104,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323152,"statfs":{"total":96636764160,"available":95562907648,"internally_reserved":1073741824,"allocated":106496,"data_stored":30023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.304}]}]}]}} 2020-04-10T07:04:22.207 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:22.210 INFO:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 --log-early pg dump --format=json 2020-04-10T07:04:22.511 INFO:teuthology.orchestra.run.smithi101.stdout: 2020-04-10T07:04:22.513 INFO:teuthology.orchestra.run.smithi101.stderr:dumped all 2020-04-10T07:04:22.523 INFO:teuthology.orchestra.run.smithi101.stdout:{"pg_ready":true,"pg_map":{"version":16,"stamp":"2020-04-10T07:04:20.596333+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":3146072,"kb_used_data":320,"kb_used_omap":0,"kb_used_meta":3145728,"kb_avail":279969448,"statfs":{"total":289910292480,"available":286688714752,"internally_reserved":3221225472,"allocated":327680,"data_stored":93582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":3221225472},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.661734"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":"8","reported_epoch":"12","state":"active+clean","last_fresh":"2020-04-10T07:04:16.950988+0000","last_change":"2020-04-10T07:04:16.950988+0000","last_active":"2020-04-10T07:04:16.950988+0000","last_peered":"2020-04-10T07:04:16.950988+0000","last_clean":"2020-04-10T07:04:16.950988+0000","last_became_active":"2020-04-10T07:04:16.950572+0000","last_became_peered":"2020-04-10T07:04:16.950572+0000","last_unstale":"2020-04-10T07:04:16.950988+0000","last_undegraded":"2020-04-10T07:04:16.950988+0000","last_fullsized":"2020-04-10T07:04:16.950988+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-04-10T07:04:15.930368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-04-10T07:04:15.930368+0000","last_clean_scrub_stamp":"2020-04-10T07:04:15.930368+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"8","reported_epoch":"12","state":"active+clean","last_fresh":"2020-04-10T07:04:16.950009+0000","last_change":"2020-04-10T07:04:16.950009+0000","last_active":"2020-04-10T07:04:16.950009+0000","last_peered":"2020-04-10T07:04:16.950009+0000","last_clean":"2020-04-10T07:04:16.950009+0000","last_became_active":"2020-04-10T07:04:16.949586+0000","last_became_peered":"2020-04-10T07:04:16.949586+0000","last_unstale":"2020-04-10T07:04:16.950009+0000","last_undegraded":"2020-04-10T07:04:16.950009+0000","last_fullsized":"2020-04-10T07:04:16.950009+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-04-10T07:04:15.930368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-04-10T07:04:15.930368+0000","last_clean_scrub_stamp":"2020-04-10T07:04:15.930368+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"8","reported_epoch":"12","state":"active+clean","last_fresh":"2020-04-10T07:04:16.948903+0000","last_change":"2020-04-10T07:04:16.948903+0000","last_active":"2020-04-10T07:04:16.948903+0000","last_peered":"2020-04-10T07:04:16.948903+0000","last_clean":"2020-04-10T07:04:16.948903+0000","last_became_active":"2020-04-10T07:04:16.948453+0000","last_became_peered":"2020-04-10T07:04:16.948453+0000","last_unstale":"2020-04-10T07:04:16.948903+0000","last_undegraded":"2020-04-10T07:04:16.948903+0000","last_fullsized":"2020-04-10T07:04:16.948903+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-04-10T07:04:15.930368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-04-10T07:04:15.930368+0000","last_clean_scrub_stamp":"2020-04-10T07:04:15.930368+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"8","reported_epoch":"12","state":"active+clean","last_fresh":"2020-04-10T07:04:16.945786+0000","last_change":"2020-04-10T07:04:16.945786+0000","last_active":"2020-04-10T07:04:16.945786+0000","last_peered":"2020-04-10T07:04:16.945786+0000","last_clean":"2020-04-10T07:04:16.945786+0000","last_became_active":"2020-04-10T07:04:16.945274+0000","last_became_peered":"2020-04-10T07:04:16.945274+0000","last_unstale":"2020-04-10T07:04:16.945786+0000","last_undegraded":"2020-04-10T07:04:16.945786+0000","last_fullsized":"2020-04-10T07:04:16.945786+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-04-10T07:04:15.930368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-04-10T07:04:15.930368+0000","last_clean_scrub_stamp":"2020-04-10T07:04:15.930368+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"9'1","reported_seq":"12","reported_epoch":"11","state":"active+clean","last_fresh":"2020-04-10T07:04:15.941720+0000","last_change":"2020-04-10T07:04:12.950377+0000","last_active":"2020-04-10T07:04:15.941720+0000","last_peered":"2020-04-10T07:04:15.941720+0000","last_clean":"2020-04-10T07:04:15.941720+0000","last_became_active":"2020-04-10T07:04:12.948567+0000","last_became_peered":"2020-04-10T07:04:12.948567+0000","last_unstale":"2020-04-10T07:04:15.941720+0000","last_undegraded":"2020-04-10T07:04:15.941720+0000","last_fullsized":"2020-04-10T07:04:15.941720+0000","mapping_epoch":8,"log_start":"0'0","ondisk_log_start":"0'0","created":6,"last_epoch_clean":9,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-04-10T07:04:09.589146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-04-10T07:04:09.589146+0000","last_clean_scrub_stamp":"2020-04-10T07:04:09.589146+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"8","reported_epoch":"12","state":"active+clean","last_fresh":"2020-04-10T07:04:16.947141+0000","last_change":"2020-04-10T07:04:16.947141+0000","last_active":"2020-04-10T07:04:16.947141+0000","last_peered":"2020-04-10T07:04:16.947141+0000","last_clean":"2020-04-10T07:04:16.947141+0000","last_became_active":"2020-04-10T07:04:16.946721+0000","last_became_peered":"2020-04-10T07:04:16.946721+0000","last_unstale":"2020-04-10T07:04:16.947141+0000","last_undegraded":"2020-04-10T07:04:16.947141+0000","last_fullsized":"2020-04-10T07:04:16.947141+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-04-10T07:04:15.930368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-04-10T07:04:15.930368+0000","last_clean_scrub_stamp":"2020-04-10T07:04:15.930368+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"8","reported_epoch":"12","state":"active+clean","last_fresh":"2020-04-10T07:04:16.949437+0000","last_change":"2020-04-10T07:04:16.949437+0000","last_active":"2020-04-10T07:04:16.949437+0000","last_peered":"2020-04-10T07:04:16.949437+0000","last_clean":"2020-04-10T07:04:16.949437+0000","last_became_active":"2020-04-10T07:04:16.949010+0000","last_became_peered":"2020-04-10T07:04:16.949010+0000","last_unstale":"2020-04-10T07:04:16.949437+0000","last_undegraded":"2020-04-10T07:04:16.949437+0000","last_fullsized":"2020-04-10T07:04:16.949437+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-04-10T07:04:15.930368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-04-10T07:04:15.930368+0000","last_clean_scrub_stamp":"2020-04-10T07:04:15.930368+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"9","reported_epoch":"12","state":"active+clean","last_fresh":"2020-04-10T07:04:16.951206+0000","last_change":"2020-04-10T07:04:16.950524+0000","last_active":"2020-04-10T07:04:16.951206+0000","last_peered":"2020-04-10T07:04:16.951206+0000","last_clean":"2020-04-10T07:04:16.951206+0000","last_became_active":"2020-04-10T07:04:16.950148+0000","last_became_peered":"2020-04-10T07:04:16.950148+0000","last_unstale":"2020-04-10T07:04:16.951206+0000","last_undegraded":"2020-04-10T07:04:16.951206+0000","last_fullsized":"2020-04-10T07:04:16.951206+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-04-10T07:04:15.930368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-04-10T07:04:15.930368+0000","last_clean_scrub_stamp":"2020-04-10T07:04:15.930368+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"8","reported_epoch":"12","state":"active+clean","last_fresh":"2020-04-10T07:04:16.947801+0000","last_change":"2020-04-10T07:04:16.947801+0000","last_active":"2020-04-10T07:04:16.947801+0000","last_peered":"2020-04-10T07:04:16.947801+0000","last_clean":"2020-04-10T07:04:16.947801+0000","last_became_active":"2020-04-10T07:04:16.947324+0000","last_became_peered":"2020-04-10T07:04:16.947324+0000","last_unstale":"2020-04-10T07:04:16.947801+0000","last_undegraded":"2020-04-10T07:04:16.947801+0000","last_fullsized":"2020-04-10T07:04:16.947801+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-04-10T07:04:15.930368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-04-10T07:04:15.930368+0000","last_clean_scrub_stamp":"2020-04-10T07:04:15.930368+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738371,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048688,"kb_used_data":104,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323152,"statfs":{"total":96636764160,"available":95562907648,"internally_reserved":1073741824,"allocated":106496,"data_stored":30023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.196}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5649999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738371,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048696,"kb_used_data":112,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323144,"statfs":{"total":96636764160,"available":95562899456,"internally_reserved":1073741824,"allocated":114688,"data_stored":33536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0169999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3220000000000001}]}]},{"osd":1,"up_from":8,"seq":34359738371,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048688,"kb_used_data":104,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323152,"statfs":{"total":96636764160,"available":95562907648,"internally_reserved":1073741824,"allocated":106496,"data_stored":30023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.304}]}]}]}} 2020-04-10T07:04:22.525 INFO:tasks.ceph.ceph_manager.cluster1:clean! 2020-04-10T07:04:22.525 INFO:tasks.ceph:Waiting until ceph cluster cluster1 is healthy... 2020-04-10T07:04:22.525 INFO:tasks.ceph.ceph_manager.cluster1:wait_until_healthy 2020-04-10T07:04:22.525 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:22.531 INFO:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 --log-early health --format=json 2020-04-10T07:04:22.880 INFO:teuthology.orchestra.run.smithi101.stdout: 2020-04-10T07:04:22.898 INFO:teuthology.orchestra.run.smithi101.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2020-04-10T07:04:22.899 INFO:tasks.ceph.ceph_manager.cluster1:wait_until_healthy done 2020-04-10T07:04:22.899 INFO:teuthology.run_tasks:Running task ceph... 2020-04-10T07:04:22.912 INFO:tasks.ceph:Creating ceph cluster cluster2... 2020-04-10T07:04:22.912 INFO:tasks.ceph:config {'mon_bind_addrvec': True, 'fs': 'xfs', 'log_whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME'], 'skip_mgr_daemons': False, 'cpu_profile': set([]), 'mkfs_options': None, 'cluster': 'cluster2', 'mon_bind_msgr2': True, 'mount_options': None, 'conf': {'client.mirror.5': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.4': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.6': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.1': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.0': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.3': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.2': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'global': {'ms inject socket failures': 5000, 'mon client directed command retry': 5}, 'mgr': {'debug ms': 1, 'debug mgr': 20}, 'client': {'debug rbd': 20, 'rbd default features': 125, 'debug rbd_mirror': 15, 'log to stderr': False}, 'mon': {'debug mon': 20, 'debug paxos': 20, 'debug ms': 1}, 'osd': {'mon osd full ratio': 0.9, 'debug ms': 20, 'bluestore fsck on mount': True, 'debug osd': 25, 'debug bluestore': 20, 'debug bluefs': 20, 'mon osd backfillfull_ratio': 0.85, 'debug rocksdb': 10, 'debug journal': 20, 'mon osd nearfull ratio': 0.8, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'bdev async discard': True, 'bluestore block size': 96636764160, 'debug filestore': 20, 'bdev enable discard': True, 'osd failsafe full ratio': 0.95}}} 2020-04-10T07:04:22.912 INFO:tasks.ceph:ctx.config {'kernel': {'sha1': 'distro', 'kdb': True}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.lock_machines': (2, 'smithi')}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'sha1': 'distro', 'kdb': True}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'sha1': '39357d2a782a02d1ea09c76cbe7bc6f222a1127a', 'extra_packages': ['rbd-mirror']}}, {'ceph': {'cluster': 'cluster1', 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME'], 'fs': 'xfs', 'conf': {'client.mirror.5': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.4': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.6': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.1': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.0': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.3': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.2': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'global': {'ms inject socket failures': 5000, 'mon client directed command retry': 5}, 'mgr': {'debug ms': 1, 'debug mgr': 20}, 'client': {'debug rbd': 20, 'rbd default features': 125, 'debug rbd_mirror': 15, 'log to stderr': False}, 'mon': {'debug mon': 20, 'debug paxos': 20, 'debug ms': 1}, 'osd': {'mon osd full ratio': 0.9, 'debug ms': 20, 'bluestore fsck on mount': True, 'debug osd': 25, 'debug bluestore': 20, 'debug bluefs': 20, 'mon osd backfillfull_ratio': 0.85, 'debug rocksdb': 10, 'debug journal': 20, 'mon osd nearfull ratio': 0.8, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'bdev async discard': True, 'bluestore block size': 96636764160, 'debug filestore': 20, 'bdev enable discard': True, 'osd failsafe full ratio': 0.95}}, 'sha1': '39357d2a782a02d1ea09c76cbe7bc6f222a1127a'}}, {'ceph': {'cluster': 'cluster2', 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME'], 'fs': 'xfs', 'conf': {'client.mirror.5': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.4': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.6': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.1': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.0': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.3': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.2': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'global': {'ms inject socket failures': 5000, 'mon client directed command retry': 5}, 'mgr': {'debug ms': 1, 'debug mgr': 20}, 'client': {'debug rbd': 20, 'rbd default features': 125, 'debug rbd_mirror': 15, 'log to stderr': False}, 'mon': {'debug mon': 20, 'debug paxos': 20, 'debug ms': 1}, 'osd': {'mon osd full ratio': 0.9, 'debug ms': 20, 'bluestore fsck on mount': True, 'debug osd': 25, 'debug bluestore': 20, 'debug bluefs': 20, 'mon osd backfillfull_ratio': 0.85, 'debug rocksdb': 10, 'debug journal': 20, 'mon osd nearfull ratio': 0.8, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'bdev async discard': True, 'bluestore block size': 96636764160, 'debug filestore': 20, 'bdev enable discard': True, 'osd failsafe full ratio': 0.95}}, 'sha1': '39357d2a782a02d1ea09c76cbe7bc6f222a1127a'}}, {'exec': {'cluster1.client.mirror.0': ["sudo ceph --cluster cluster1 auth caps client.mirror.4 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster1 auth caps client.mirror.5 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster1 auth caps client.mirror.6 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror.4 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror.5 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror.6 mon 'profile rbd-mirror' osd 'profile rbd'"]}}, {'exec': {'cluster1.client.mirror.0': ["sudo ceph --cluster cluster1 auth caps client.mirror mon 'profile rbd-mirror-peer' osd 'profile rbd'", "sudo ceph --cluster cluster1 auth caps client.mirror.0 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster1 auth caps client.mirror.1 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster1 auth caps client.mirror.2 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster1 auth caps client.mirror.3 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror mon 'profile rbd-mirror-peer' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror.0 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror.1 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror.2 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror.3 mon 'profile rbd-mirror' osd 'profile rbd'"]}}, {'workunit': {'clients': {'cluster1.client.mirror': ['rbd/rbd_mirror_snapshot.sh']}, 'env': {'CEPH_ARGS': '', 'RBD_IMAGE_FEATURES': 'layering,exclusive-lock,journaling', 'RBD_MIRROR_INSTANCES': '4', 'RBD_MIRROR_CONFIG_KEY': '1', 'RBD_MIRROR_USE_EXISTING_CLUSTER': '1'}}}], 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_octopus/qa', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/worker.smithi.2404', 'meta': [{'desc': 'run two ceph clusters and install rbd-mirror'}, {'desc': 'configure the permissions for client.mirror'}, {'desc': 'configure the permissions for client.mirror'}, {'desc': '2 ceph clusters with 1 mon and 3 osds each'}, {'desc': 'run the rbd_mirror_snapshot.sh workunit to test the rbd-mirror daemon'}], 'owner': 'scheduled_teuthology@teuthology', 'openstack': [{'volumes': {'count': 3, 'size': 30}}], 'targets': {'smithi101.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDT9Fklxr7Y5E4UHPwH/a7MBIWYJNi0zKSWKVDcl9NGrTUcSbyXl/q3mNS6XQWwWcfHi4kllR1oWNJMGjcjNMqN2Ik7xQG0CN3j6IxR/ThSBv7eyIUSAldogJUNIte/ae3MyZlpyKhMSxSII/tStu2kQ4lchx7uAYYoXPW8dg178Lr3m/alXSFlfu5TrZIhfa7lKMmYXhXrrkwWDaaX8jqpXi+SYBkAa3mWhHiMihALIKdQRsUmMaAlPYQSnVuA4iU4JIt3x9/0BXHIozs8hrUMQ7MLSjXaKmH3trGj8P/hy65A7yVd1F4RUMUUHcK4AVo/4gHlW/G3+y7vOtBUfDXJ', 'smithi204.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCSzeRmp/sjnId4Ox3+K4SEbMVfVJu6es2vZjpBuMNFfBvRZF3JaOt+F82hMD1lDOJFSzAqT/hKwcdqT5vMPkbkQVnTUTz6KWx5h0Magh8oRggNmVvOswW8Gcz8ktHBV+GSTWy8DISCgruSYDn2z57IoqFYwdf0GFBhZMX1RNs4nqCIHW3RKLCtRlD6g9P2wHktAjGcJk73J8rY66EVc4+hY/V6p67qXU1jU2mCc0ktr4U2vgIRYS4p4AZi1uSYG934vbZTMzvm4YFFt1M6ygHYNPgxF77Z22HIti6AGi3lx0S84WT2wY2NnlAbTeMzvJa8eZnARRhSv+fYJPGVcz+r'}, 'job_id': '4937628', 'tube': 'smithi', 'suite_branch': 'octopus', 'priority': 1000, 'os_version': '18.04', 'first_in_suite': False, 'branch': 'octopus', 'suite': 'rbd', 'email': 'ceph-qa@ceph.io', 'archive_path': '/home/teuthworker/archive/teuthology-2020-04-09_06:00:03-rbd-octopus-distro-basic-smithi/4937628', 'description': 'rbd/mirror/{base/install.yaml clients/{mirror-extra.yaml mirror.yaml} cluster/{2-node.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-low-osd-mem-target.yaml supported-random-distro$/{ubuntu_latest.yaml} workloads/rbd-mirror-snapshot-workunit-journaling.yaml}', 'nuke-on-error': True, 'repo': 'git://git.ceph.com/ceph.git', 'last_in_suite': False, 'machine_type': 'smithi', 'teuthology_branch': 'master', 'sha1': '39357d2a782a02d1ea09c76cbe7bc6f222a1127a', 'name': 'teuthology-2020-04-09_06:00:03-rbd-octopus-distro-basic-smithi', 'roles': [['cluster1.mon.a', 'cluster1.mgr.x', 'cluster2.mgr.x', 'cluster1.osd.0', 'cluster1.osd.1', 'cluster1.osd.2', 'cluster1.client.0', 'cluster2.client.0'], ['cluster2.mon.a', 'cluster2.osd.0', 'cluster2.osd.1', 'cluster2.osd.2', 'cluster1.client.mirror', 'cluster1.client.mirror.0', 'cluster1.client.mirror.1', 'cluster1.client.mirror.2', 'cluster1.client.mirror.3', 'cluster1.client.mirror.4', 'cluster1.client.mirror.5', 'cluster1.client.mirror.6', 'cluster2.client.mirror', 'cluster2.client.mirror.0', 'cluster2.client.mirror.1', 'cluster2.client.mirror.2', 'cluster2.client.mirror.3', 'cluster2.client.mirror.4', 'cluster2.client.mirror.5', 'cluster2.client.mirror.6']], 'suite_relpath': 'qa', 'suite_repo': 'git://git.ceph.com/ceph.git', 'overrides': {'ceph': {'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME'], 'conf': {'client.mirror.5': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.4': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.6': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.1': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.0': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.3': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.2': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'global': {'ms inject socket failures': 5000, 'mon client directed command retry': 5}, 'mgr': {'debug ms': 1, 'debug mgr': 20}, 'client': {'debug rbd': 20, 'rbd default features': 125, 'debug rbd_mirror': 15, 'log to stderr': False}, 'mon': {'debug mon': 20, 'debug paxos': 20, 'debug ms': 1}, 'osd': {'mon osd full ratio': 0.9, 'debug ms': 20, 'bluestore fsck on mount': True, 'debug osd': 25, 'debug bluestore': 20, 'debug bluefs': 20, 'mon osd backfillfull_ratio': 0.85, 'debug rocksdb': 10, 'debug journal': 20, 'mon osd nearfull ratio': 0.8, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'bdev async discard': True, 'bluestore block size': 96636764160, 'debug filestore': 20, 'bdev enable discard': True, 'osd failsafe full ratio': 0.95}}, 'fs': 'xfs', 'sha1': '39357d2a782a02d1ea09c76cbe7bc6f222a1127a'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}}}, 'install': {'ceph': {'sha1': '39357d2a782a02d1ea09c76cbe7bc6f222a1127a'}}, 'admin_socket': {'branch': 'octopus'}, 'thrashosds': {'bdev_inject_crash_probability': 0.5, 'bdev_inject_crash': 2}, 'workunit': {'sha1': '39357d2a782a02d1ea09c76cbe7bc6f222a1127a', 'branch': 'octopus'}}, 'os_type': 'ubuntu', 'suite_sha1': '39357d2a782a02d1ea09c76cbe7bc6f222a1127a'} 2020-04-10T07:04:22.913 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:22.919 INFO:teuthology.orchestra.run.smithi101:> install -d -m0755 -- /home/ubuntu/cephtest/cluster2.data 2020-04-10T07:04:22.966 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:22.972 INFO:teuthology.orchestra.run.smithi204:> install -d -m0755 -- /home/ubuntu/cephtest/cluster2.data 2020-04-10T07:04:23.021 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:23.026 INFO:teuthology.orchestra.run.smithi101:> sudo install -d -m0777 -- /var/run/ceph 2020-04-10T07:04:23.070 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:23.076 INFO:teuthology.orchestra.run.smithi204:> sudo install -d -m0777 -- /var/run/ceph 2020-04-10T07:04:23.182 DEBUG:teuthology.orchestra.remote:smithi204:/scratch_devs is 72B 2020-04-10T07:04:23.194 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_4', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_1'] 2020-04-10T07:04:23.194 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:23.200 INFO:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2020-04-10T07:04:23.249 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-1 2020-04-10T07:04:23.249 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-04-10T07:04:23.250 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 6h/6d Inode: 349 Links: 1 2020-04-10T07:04:23.250 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-04-10T07:04:23.250 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2020-04-10 07:01:13.874240416 +0000 2020-04-10T07:04:23.250 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2020-04-10 07:01:13.742306419 +0000 2020-04-10T07:04:23.250 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2020-04-10 07:01:13.742306419 +0000 2020-04-10T07:04:23.250 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: - 2020-04-10T07:04:23.259 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2020-04-10T07:04:23.259 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2020-04-10T07:04:23.260 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000384744 s, 1.3 MB/s 2020-04-10T07:04:23.264 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:23.313 INFO:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2020-04-10T07:04:23.361 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2020-04-10T07:04:23.361 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-04-10T07:04:23.361 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 6h/6d Inode: 355 Links: 1 2020-04-10T07:04:23.361 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-04-10T07:04:23.361 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2020-04-10 07:01:14.118118411 +0000 2020-04-10T07:04:23.361 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2020-04-10 07:01:13.982186414 +0000 2020-04-10T07:04:23.362 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2020-04-10 07:01:13.982186414 +0000 2020-04-10T07:04:23.362 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: - 2020-04-10T07:04:23.370 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2020-04-10T07:04:23.370 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2020-04-10T07:04:23.371 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000407568 s, 1.3 MB/s 2020-04-10T07:04:23.374 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:23.423 INFO:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2020-04-10T07:04:23.472 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-3 2020-04-10T07:04:23.473 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-04-10T07:04:23.473 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 6h/6d Inode: 361 Links: 1 2020-04-10T07:04:23.473 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-04-10T07:04:23.473 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2020-04-10 07:01:14.421966403 +0000 2020-04-10T07:04:23.473 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2020-04-10 07:01:14.262046408 +0000 2020-04-10T07:04:23.473 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2020-04-10 07:01:14.262046408 +0000 2020-04-10T07:04:23.474 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: - 2020-04-10T07:04:23.482 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2020-04-10T07:04:23.482 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2020-04-10T07:04:23.483 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000405898 s, 1.3 MB/s 2020-04-10T07:04:23.487 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:23.535 INFO:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2020-04-10T07:04:23.584 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-4 2020-04-10T07:04:23.585 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-04-10T07:04:23.585 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 6h/6d Inode: 367 Links: 1 2020-04-10T07:04:23.585 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-04-10T07:04:23.585 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2020-04-10 07:03:33.031238761 +0000 2020-04-10T07:04:23.585 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2020-04-10 07:01:14.573890401 +0000 2020-04-10T07:04:23.585 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2020-04-10 07:01:14.573890401 +0000 2020-04-10T07:04:23.585 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: - 2020-04-10T07:04:23.594 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2020-04-10T07:04:23.594 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2020-04-10T07:04:23.594 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000403274 s, 1.3 MB/s 2020-04-10T07:04:23.598 INFO:tasks.ceph:osd dev map: {'cluster2.osd.0': '/dev/vg_nvme/lv_4', 'cluster2.osd.1': '/dev/vg_nvme/lv_3', 'cluster2.osd.2': '/dev/vg_nvme/lv_2'} 2020-04-10T07:04:23.598 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi204.front.sepia.ceph.com'): {'cluster2.osd.0': '/dev/vg_nvme/lv_4', 'cluster2.osd.1': '/dev/vg_nvme/lv_3', 'cluster2.osd.2': '/dev/vg_nvme/lv_2'}} 2020-04-10T07:04:23.599 INFO:tasks.ceph:Generating config... 2020-04-10T07:04:23.600 INFO:tasks.ceph:[client.mirror.5] admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2020-04-10T07:04:23.600 INFO:tasks.ceph:[client.mirror.5] pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2020-04-10T07:04:23.600 INFO:tasks.ceph:[client.mirror.4] admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2020-04-10T07:04:23.600 INFO:tasks.ceph:[client.mirror.4] pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2020-04-10T07:04:23.601 INFO:tasks.ceph:[client.mirror.6] admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2020-04-10T07:04:23.601 INFO:tasks.ceph:[client.mirror.6] pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2020-04-10T07:04:23.601 INFO:tasks.ceph:[client.mirror.1] admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2020-04-10T07:04:23.601 INFO:tasks.ceph:[client.mirror.1] pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2020-04-10T07:04:23.601 INFO:tasks.ceph:[client.mirror.0] admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2020-04-10T07:04:23.601 INFO:tasks.ceph:[client.mirror.0] pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2020-04-10T07:04:23.602 INFO:tasks.ceph:[client.mirror.3] admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2020-04-10T07:04:23.602 INFO:tasks.ceph:[client.mirror.3] pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2020-04-10T07:04:23.602 INFO:tasks.ceph:[client.mirror.2] admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2020-04-10T07:04:23.602 INFO:tasks.ceph:[client.mirror.2] pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2020-04-10T07:04:23.602 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2020-04-10T07:04:23.602 INFO:tasks.ceph:[global] mon client directed command retry = 5 2020-04-10T07:04:23.602 INFO:tasks.ceph:[mgr] debug ms = 1 2020-04-10T07:04:23.603 INFO:tasks.ceph:[mgr] debug mgr = 20 2020-04-10T07:04:23.603 INFO:tasks.ceph:[client] debug rbd = 20 2020-04-10T07:04:23.603 INFO:tasks.ceph:[client] rbd default features = 125 2020-04-10T07:04:23.603 INFO:tasks.ceph:[client] debug rbd_mirror = 15 2020-04-10T07:04:23.603 INFO:tasks.ceph:[client] log to stderr = False 2020-04-10T07:04:23.603 INFO:tasks.ceph:[mon] debug mon = 20 2020-04-10T07:04:23.603 INFO:tasks.ceph:[mon] debug paxos = 20 2020-04-10T07:04:23.604 INFO:tasks.ceph:[mon] debug ms = 1 2020-04-10T07:04:23.604 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2020-04-10T07:04:23.604 INFO:tasks.ceph:[osd] debug ms = 20 2020-04-10T07:04:23.604 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2020-04-10T07:04:23.604 INFO:tasks.ceph:[osd] debug osd = 25 2020-04-10T07:04:23.604 INFO:tasks.ceph:[osd] debug bluestore = 20 2020-04-10T07:04:23.604 INFO:tasks.ceph:[osd] debug bluefs = 20 2020-04-10T07:04:23.605 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2020-04-10T07:04:23.605 INFO:tasks.ceph:[osd] debug rocksdb = 10 2020-04-10T07:04:23.605 INFO:tasks.ceph:[osd] debug journal = 20 2020-04-10T07:04:23.605 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2020-04-10T07:04:23.605 INFO:tasks.ceph:[osd] osd memory target = 1610612736 2020-04-10T07:04:23.605 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2020-04-10T07:04:23.605 INFO:tasks.ceph:[osd] bdev async discard = True 2020-04-10T07:04:23.605 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2020-04-10T07:04:23.606 INFO:tasks.ceph:[osd] debug filestore = 20 2020-04-10T07:04:23.606 INFO:tasks.ceph:[osd] bdev enable discard = True 2020-04-10T07:04:23.606 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2020-04-10T07:04:23.606 INFO:tasks.ceph:Setting up cluster2.mon.a... 2020-04-10T07:04:23.606 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:23.647 INFO:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/cluster2.keyring 2020-04-10T07:04:23.721 INFO:teuthology.orchestra.run.smithi204.stdout:creating /etc/ceph/cluster2.keyring 2020-04-10T07:04:23.723 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:23.771 INFO:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/cluster2.keyring 2020-04-10T07:04:23.849 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:23.895 INFO:teuthology.orchestra.run.smithi204:> sudo chmod 0644 /etc/ceph/cluster2.keyring 2020-04-10T07:04:23.952 DEBUG:tasks.ceph:Ceph mon addresses: [('cluster2.mon.a', '172.21.15.204')] 2020-04-10T07:04:23.952 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:23.999 INFO:teuthology.orchestra.run.smithi204:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool --create --clobber --enable-all-features --add a 172.21.15.204 --print /home/ubuntu/cephtest/cluster2.monmap 2020-04-10T07:04:24.069 INFO:teuthology.orchestra.run.smithi204.stdout:monmaptool: monmap file /home/ubuntu/cephtest/cluster2.monmap 2020-04-10T07:04:24.069 INFO:teuthology.orchestra.run.smithi204.stdout:monmaptool: generated fsid b4476b22-7943-481f-a1ec-34bc78d4adbc 2020-04-10T07:04:24.069 INFO:teuthology.orchestra.run.smithi204.stdout:epoch 0 2020-04-10T07:04:24.069 INFO:teuthology.orchestra.run.smithi204.stdout:fsid b4476b22-7943-481f-a1ec-34bc78d4adbc 2020-04-10T07:04:24.070 INFO:teuthology.orchestra.run.smithi204.stdout:last_changed 2020-04-10T07:04:24.070160+0000 2020-04-10T07:04:24.070 INFO:teuthology.orchestra.run.smithi204.stdout:created 2020-04-10T07:04:24.070160+0000 2020-04-10T07:04:24.070 INFO:teuthology.orchestra.run.smithi204.stdout:min_mon_release 0 (unknown) 2020-04-10T07:04:24.070 INFO:teuthology.orchestra.run.smithi204.stdout:0: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.a 2020-04-10T07:04:24.070 INFO:teuthology.orchestra.run.smithi204.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/cluster2.monmap (1 monitors) 2020-04-10T07:04:24.071 INFO:tasks.ceph:Writing /etc/ceph/cluster2.conf for FSID b4476b22-7943-481f-a1ec-34bc78d4adbc... 2020-04-10T07:04:24.072 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:24.078 INFO:teuthology.orchestra.run.smithi101:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/cluster2.conf && sudo chmod 0644 /etc/ceph/cluster2.conf > /dev/null 2020-04-10T07:04:24.122 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:24.128 INFO:teuthology.orchestra.run.smithi204:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/cluster2.conf && sudo chmod 0644 /etc/ceph/cluster2.conf > /dev/null 2020-04-10T07:04:24.173 INFO:teuthology.orchestra.run.smithi101.stdout:[global] 2020-04-10T07:04:24.173 INFO:teuthology.orchestra.run.smithi101.stdout: chdir = "" 2020-04-10T07:04:24.174 INFO:teuthology.orchestra.run.smithi101.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-04-10T07:04:24.174 INFO:teuthology.orchestra.run.smithi101.stdout: auth supported = cephx 2020-04-10T07:04:24.174 INFO:teuthology.orchestra.run.smithi101.stdout: 2020-04-10T07:04:24.174 INFO:teuthology.orchestra.run.smithi101.stdout: filestore xattr use omap = true 2020-04-10T07:04:24.174 INFO:teuthology.orchestra.run.smithi101.stdout: 2020-04-10T07:04:24.174 INFO:teuthology.orchestra.run.smithi101.stdout: mon clock drift allowed = 1.000 2020-04-10T07:04:24.174 INFO:teuthology.orchestra.run.smithi101.stdout: 2020-04-10T07:04:24.174 INFO:teuthology.orchestra.run.smithi101.stdout: osd crush chooseleaf type = 0 2020-04-10T07:04:24.175 INFO:teuthology.orchestra.run.smithi101.stdout: auth debug = true 2020-04-10T07:04:24.175 INFO:teuthology.orchestra.run.smithi101.stdout: 2020-04-10T07:04:24.175 INFO:teuthology.orchestra.run.smithi101.stdout: ms die on old message = true 2020-04-10T07:04:24.175 INFO:teuthology.orchestra.run.smithi101.stdout: ms die on bug = true 2020-04-10T07:04:24.175 INFO:teuthology.orchestra.run.smithi101.stdout: 2020-04-10T07:04:24.175 INFO:teuthology.orchestra.run.smithi101.stdout: mon max pg per osd = 10000 # >= luminous 2020-04-10T07:04:24.175 INFO:teuthology.orchestra.run.smithi101.stdout: mon pg warn max object skew = 0 2020-04-10T07:04:24.176 INFO:teuthology.orchestra.run.smithi101.stdout: 2020-04-10T07:04:24.176 INFO:teuthology.orchestra.run.smithi101.stdout: # disable pg_autoscaler by default for new pools 2020-04-10T07:04:24.176 INFO:teuthology.orchestra.run.smithi101.stdout: osd_pool_default_pg_autoscale_mode = off 2020-04-10T07:04:24.176 INFO:teuthology.orchestra.run.smithi101.stdout: 2020-04-10T07:04:24.176 INFO:teuthology.orchestra.run.smithi101.stdout: osd pool default size = 2 2020-04-10T07:04:24.176 INFO:teuthology.orchestra.run.smithi101.stdout: 2020-04-10T07:04:24.176 INFO:teuthology.orchestra.run.smithi101.stdout: mon osd allow primary affinity = true 2020-04-10T07:04:24.177 INFO:teuthology.orchestra.run.smithi101.stdout: mon osd allow pg remap = true 2020-04-10T07:04:24.177 INFO:teuthology.orchestra.run.smithi101.stdout: mon warn on legacy crush tunables = false 2020-04-10T07:04:24.177 INFO:teuthology.orchestra.run.smithi101.stdout: mon warn on crush straw calc version zero = false 2020-04-10T07:04:24.177 INFO:teuthology.orchestra.run.smithi101.stdout: mon warn on no sortbitwise = false 2020-04-10T07:04:24.177 INFO:teuthology.orchestra.run.smithi101.stdout: mon warn on osd down out interval zero = false 2020-04-10T07:04:24.177 INFO:teuthology.orchestra.run.smithi101.stdout: mon warn on too few osds = false 2020-04-10T07:04:24.177 INFO:teuthology.orchestra.run.smithi101.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-04-10T07:04:24.178 INFO:teuthology.orchestra.run.smithi101.stdout: mon_warn_on_pool_no_redundancy = false 2020-04-10T07:04:24.178 INFO:teuthology.orchestra.run.smithi101.stdout: 2020-04-10T07:04:24.178 INFO:teuthology.orchestra.run.smithi101.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-04-10T07:04:24.178 INFO:teuthology.orchestra.run.smithi101.stdout: 2020-04-10T07:04:24.178 INFO:teuthology.orchestra.run.smithi101.stdout: osd default data pool replay window = 5 2020-04-10T07:04:24.178 INFO:teuthology.orchestra.run.smithi101.stdout: 2020-04-10T07:04:24.178 INFO:teuthology.orchestra.run.smithi101.stdout: mon allow pool delete = true 2020-04-10T07:04:24.178 INFO:teuthology.orchestra.run.smithi101.stdout: 2020-04-10T07:04:24.179 INFO:teuthology.orchestra.run.smithi101.stdout: mon cluster log file level = debug 2020-04-10T07:04:24.179 INFO:teuthology.orchestra.run.smithi101.stdout: debug asserts on shutdown = true 2020-04-10T07:04:24.179 INFO:teuthology.orchestra.run.smithi101.stdout: mon host = 172.21.15.204 2020-04-10T07:04:24.179 INFO:teuthology.orchestra.run.smithi101.stdout: ms inject socket failures = 5000 2020-04-10T07:04:24.179 INFO:teuthology.orchestra.run.smithi101.stdout: mon client directed command retry = 5 2020-04-10T07:04:24.179 INFO:teuthology.orchestra.run.smithi101.stdout: fsid = b4476b22-7943-481f-a1ec-34bc78d4adbc 2020-04-10T07:04:24.179 INFO:teuthology.orchestra.run.smithi101.stdout: 2020-04-10T07:04:24.180 INFO:teuthology.orchestra.run.smithi101.stdout:[osd] 2020-04-10T07:04:24.180 INFO:teuthology.orchestra.run.smithi101.stdout: osd journal size = 100 2020-04-10T07:04:24.180 INFO:teuthology.orchestra.run.smithi101.stdout: 2020-04-10T07:04:24.180 INFO:teuthology.orchestra.run.smithi101.stdout: osd scrub load threshold = 5.0 2020-04-10T07:04:24.180 INFO:teuthology.orchestra.run.smithi101.stdout: osd scrub max interval = 600 2020-04-10T07:04:24.180 INFO:teuthology.orchestra.run.smithi101.stdout: 2020-04-10T07:04:24.180 INFO:teuthology.orchestra.run.smithi101.stdout: osd recover clone overlap = true 2020-04-10T07:04:24.181 INFO:teuthology.orchestra.run.smithi101.stdout: osd recovery max chunk = 1048576 2020-04-10T07:04:24.182 INFO:teuthology.orchestra.run.smithi101.stdout: 2020-04-10T07:04:24.182 INFO:teuthology.orchestra.run.smithi101.stdout: osd debug shutdown = true 2020-04-10T07:04:24.182 INFO:teuthology.orchestra.run.smithi101.stdout: osd debug op order = true 2020-04-10T07:04:24.182 INFO:teuthology.orchestra.run.smithi101.stdout: osd debug verify stray on activate = true 2020-04-10T07:04:24.183 INFO:teuthology.orchestra.run.smithi101.stdout: 2020-04-10T07:04:24.183 INFO:teuthology.orchestra.run.smithi101.stdout: osd open classes on start = true 2020-04-10T07:04:24.183 INFO:teuthology.orchestra.run.smithi101.stdout: osd debug pg log writeout = true 2020-04-10T07:04:24.183 INFO:teuthology.orchestra.run.smithi101.stdout: 2020-04-10T07:04:24.183 INFO:teuthology.orchestra.run.smithi101.stdout: osd deep scrub update digest min age = 30 2020-04-10T07:04:24.184 INFO:teuthology.orchestra.run.smithi101.stdout: 2020-04-10T07:04:24.184 INFO:teuthology.orchestra.run.smithi101.stdout: osd map max advance = 10 2020-04-10T07:04:24.184 INFO:teuthology.orchestra.run.smithi101.stdout: 2020-04-10T07:04:24.184 INFO:teuthology.orchestra.run.smithi101.stdout: journal zero on create = true 2020-04-10T07:04:24.184 INFO:teuthology.orchestra.run.smithi101.stdout: 2020-04-10T07:04:24.184 INFO:teuthology.orchestra.run.smithi101.stdout: filestore ondisk finisher threads = 3 2020-04-10T07:04:24.185 INFO:teuthology.orchestra.run.smithi101.stdout: filestore apply finisher threads = 3 2020-04-10T07:04:24.185 INFO:teuthology.orchestra.run.smithi101.stdout: 2020-04-10T07:04:24.185 INFO:teuthology.orchestra.run.smithi101.stdout: bdev debug aio = true 2020-04-10T07:04:24.186 INFO:teuthology.orchestra.run.smithi101.stdout: osd debug misdirected ops = true 2020-04-10T07:04:24.186 INFO:teuthology.orchestra.run.smithi101.stdout: mon osd full ratio = 0.9 2020-04-10T07:04:24.186 INFO:teuthology.orchestra.run.smithi101.stdout: debug ms = 20 2020-04-10T07:04:24.186 INFO:teuthology.orchestra.run.smithi101.stdout: bluestore fsck on mount = True 2020-04-10T07:04:24.186 INFO:teuthology.orchestra.run.smithi101.stdout: debug osd = 25 2020-04-10T07:04:24.186 INFO:teuthology.orchestra.run.smithi101.stdout: debug bluestore = 20 2020-04-10T07:04:24.186 INFO:teuthology.orchestra.run.smithi101.stdout: debug bluefs = 20 2020-04-10T07:04:24.187 INFO:teuthology.orchestra.run.smithi101.stdout: mon osd backfillfull_ratio = 0.85 2020-04-10T07:04:24.187 INFO:teuthology.orchestra.run.smithi101.stdout: debug rocksdb = 10 2020-04-10T07:04:24.187 INFO:teuthology.orchestra.run.smithi101.stdout: debug journal = 20 2020-04-10T07:04:24.187 INFO:teuthology.orchestra.run.smithi101.stdout: mon osd nearfull ratio = 0.8 2020-04-10T07:04:24.187 INFO:teuthology.orchestra.run.smithi101.stdout: osd memory target = 1610612736 2020-04-10T07:04:24.187 INFO:teuthology.orchestra.run.smithi101.stdout: osd objectstore = bluestore 2020-04-10T07:04:24.187 INFO:teuthology.orchestra.run.smithi101.stdout: bdev async discard = True 2020-04-10T07:04:24.188 INFO:teuthology.orchestra.run.smithi101.stdout: bluestore block size = 96636764160 2020-04-10T07:04:24.188 INFO:teuthology.orchestra.run.smithi101.stdout: debug filestore = 20 2020-04-10T07:04:24.188 INFO:teuthology.orchestra.run.smithi101.stdout: bdev enable discard = True 2020-04-10T07:04:24.188 INFO:teuthology.orchestra.run.smithi101.stdout: osd failsafe full ratio = 0.95 2020-04-10T07:04:24.188 INFO:teuthology.orchestra.run.smithi101.stdout: 2020-04-10T07:04:24.188 INFO:teuthology.orchestra.run.smithi101.stdout:[mgr] 2020-04-10T07:04:24.188 INFO:teuthology.orchestra.run.smithi101.stdout: debug ms = 1 2020-04-10T07:04:24.189 INFO:teuthology.orchestra.run.smithi101.stdout: debug mgr = 20 2020-04-10T07:04:24.189 INFO:teuthology.orchestra.run.smithi101.stdout: debug mon = 20 2020-04-10T07:04:24.189 INFO:teuthology.orchestra.run.smithi101.stdout: debug auth = 20 2020-04-10T07:04:24.189 INFO:teuthology.orchestra.run.smithi101.stdout: mon reweight min pgs per osd = 4 2020-04-10T07:04:24.189 INFO:teuthology.orchestra.run.smithi101.stdout: mon reweight min bytes per osd = 10 2020-04-10T07:04:24.190 INFO:teuthology.orchestra.run.smithi101.stdout: mgr/telemetry/nag = false 2020-04-10T07:04:24.190 INFO:teuthology.orchestra.run.smithi101.stdout: 2020-04-10T07:04:24.190 INFO:teuthology.orchestra.run.smithi101.stdout:[mon] 2020-04-10T07:04:24.190 INFO:teuthology.orchestra.run.smithi101.stdout: debug ms = 1 2020-04-10T07:04:24.190 INFO:teuthology.orchestra.run.smithi101.stdout: debug mon = 20 2020-04-10T07:04:24.191 INFO:teuthology.orchestra.run.smithi101.stdout: debug paxos = 20 2020-04-10T07:04:24.191 INFO:teuthology.orchestra.run.smithi101.stdout: debug auth = 20 2020-04-10T07:04:24.191 INFO:teuthology.orchestra.run.smithi101.stdout: mon data avail warn = 5 2020-04-10T07:04:24.191 INFO:teuthology.orchestra.run.smithi101.stdout: mon mgr mkfs grace = 240 2020-04-10T07:04:24.192 INFO:teuthology.orchestra.run.smithi101.stdout: mon reweight min pgs per osd = 4 2020-04-10T07:04:24.192 INFO:teuthology.orchestra.run.smithi101.stdout: mon osd reporter subtree level = osd 2020-04-10T07:04:24.192 INFO:teuthology.orchestra.run.smithi101.stdout: mon osd prime pg temp = true 2020-04-10T07:04:24.192 INFO:teuthology.orchestra.run.smithi101.stdout: mon reweight min bytes per osd = 10 2020-04-10T07:04:24.192 INFO:teuthology.orchestra.run.smithi101.stdout: 2020-04-10T07:04:24.193 INFO:teuthology.orchestra.run.smithi101.stdout:[client] 2020-04-10T07:04:24.193 INFO:teuthology.orchestra.run.smithi101.stdout: rgw cache enabled = true 2020-04-10T07:04:24.193 INFO:teuthology.orchestra.run.smithi101.stdout: rgw enable ops log = true 2020-04-10T07:04:24.193 INFO:teuthology.orchestra.run.smithi101.stdout: rgw enable usage log = true 2020-04-10T07:04:24.193 INFO:teuthology.orchestra.run.smithi101.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-04-10T07:04:24.193 INFO:teuthology.orchestra.run.smithi101.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-04-10T07:04:24.194 INFO:teuthology.orchestra.run.smithi101.stdout: debug rbd = 20 2020-04-10T07:04:24.194 INFO:teuthology.orchestra.run.smithi101.stdout: rbd default features = 125 2020-04-10T07:04:24.194 INFO:teuthology.orchestra.run.smithi101.stdout: debug rbd_mirror = 15 2020-04-10T07:04:24.194 INFO:teuthology.orchestra.run.smithi101.stdout: log to stderr = False 2020-04-10T07:04:24.194 INFO:teuthology.orchestra.run.smithi101.stdout:[mon.a] 2020-04-10T07:04:24.194 INFO:teuthology.orchestra.run.smithi101.stdout:[client.mirror.5] 2020-04-10T07:04:24.194 INFO:teuthology.orchestra.run.smithi101.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2020-04-10T07:04:24.195 INFO:teuthology.orchestra.run.smithi101.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2020-04-10T07:04:24.195 INFO:teuthology.orchestra.run.smithi101.stdout:[client.mirror.4] 2020-04-10T07:04:24.195 INFO:teuthology.orchestra.run.smithi101.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2020-04-10T07:04:24.195 INFO:teuthology.orchestra.run.smithi101.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2020-04-10T07:04:24.195 INFO:teuthology.orchestra.run.smithi101.stdout:[client.mirror.6] 2020-04-10T07:04:24.195 INFO:teuthology.orchestra.run.smithi101.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2020-04-10T07:04:24.195 INFO:teuthology.orchestra.run.smithi101.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2020-04-10T07:04:24.196 INFO:teuthology.orchestra.run.smithi101.stdout:[client.mirror.1] 2020-04-10T07:04:24.196 INFO:teuthology.orchestra.run.smithi101.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2020-04-10T07:04:24.196 INFO:teuthology.orchestra.run.smithi101.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2020-04-10T07:04:24.196 INFO:teuthology.orchestra.run.smithi101.stdout:[client.mirror.0] 2020-04-10T07:04:24.196 INFO:teuthology.orchestra.run.smithi101.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2020-04-10T07:04:24.196 INFO:teuthology.orchestra.run.smithi101.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2020-04-10T07:04:24.196 INFO:teuthology.orchestra.run.smithi101.stdout:[client.mirror.3] 2020-04-10T07:04:24.197 INFO:teuthology.orchestra.run.smithi101.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2020-04-10T07:04:24.197 INFO:teuthology.orchestra.run.smithi101.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2020-04-10T07:04:24.197 INFO:teuthology.orchestra.run.smithi101.stdout:[client.mirror.2] 2020-04-10T07:04:24.197 INFO:teuthology.orchestra.run.smithi101.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2020-04-10T07:04:24.197 INFO:teuthology.orchestra.run.smithi101.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2020-04-10T07:04:24.202 INFO:teuthology.orchestra.run.smithi204.stdout:[global] 2020-04-10T07:04:24.202 INFO:teuthology.orchestra.run.smithi204.stdout: chdir = "" 2020-04-10T07:04:24.202 INFO:teuthology.orchestra.run.smithi204.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-04-10T07:04:24.202 INFO:teuthology.orchestra.run.smithi204.stdout: auth supported = cephx 2020-04-10T07:04:24.202 INFO:teuthology.orchestra.run.smithi204.stdout: 2020-04-10T07:04:24.202 INFO:teuthology.orchestra.run.smithi204.stdout: filestore xattr use omap = true 2020-04-10T07:04:24.203 INFO:teuthology.orchestra.run.smithi204.stdout: 2020-04-10T07:04:24.203 INFO:teuthology.orchestra.run.smithi204.stdout: mon clock drift allowed = 1.000 2020-04-10T07:04:24.203 INFO:teuthology.orchestra.run.smithi204.stdout: 2020-04-10T07:04:24.203 INFO:teuthology.orchestra.run.smithi204.stdout: osd crush chooseleaf type = 0 2020-04-10T07:04:24.203 INFO:teuthology.orchestra.run.smithi204.stdout: auth debug = true 2020-04-10T07:04:24.203 INFO:teuthology.orchestra.run.smithi204.stdout: 2020-04-10T07:04:24.203 INFO:teuthology.orchestra.run.smithi204.stdout: ms die on old message = true 2020-04-10T07:04:24.203 INFO:teuthology.orchestra.run.smithi204.stdout: ms die on bug = true 2020-04-10T07:04:24.204 INFO:teuthology.orchestra.run.smithi204.stdout: 2020-04-10T07:04:24.204 INFO:teuthology.orchestra.run.smithi204.stdout: mon max pg per osd = 10000 # >= luminous 2020-04-10T07:04:24.204 INFO:teuthology.orchestra.run.smithi204.stdout: mon pg warn max object skew = 0 2020-04-10T07:04:24.205 INFO:teuthology.orchestra.run.smithi204.stdout: 2020-04-10T07:04:24.205 INFO:teuthology.orchestra.run.smithi204.stdout: # disable pg_autoscaler by default for new pools 2020-04-10T07:04:24.205 INFO:teuthology.orchestra.run.smithi204.stdout: osd_pool_default_pg_autoscale_mode = off 2020-04-10T07:04:24.205 INFO:teuthology.orchestra.run.smithi204.stdout: 2020-04-10T07:04:24.205 INFO:teuthology.orchestra.run.smithi204.stdout: osd pool default size = 2 2020-04-10T07:04:24.205 INFO:teuthology.orchestra.run.smithi204.stdout: 2020-04-10T07:04:24.206 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd allow primary affinity = true 2020-04-10T07:04:24.206 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd allow pg remap = true 2020-04-10T07:04:24.206 INFO:teuthology.orchestra.run.smithi204.stdout: mon warn on legacy crush tunables = false 2020-04-10T07:04:24.206 INFO:teuthology.orchestra.run.smithi204.stdout: mon warn on crush straw calc version zero = false 2020-04-10T07:04:24.206 INFO:teuthology.orchestra.run.smithi204.stdout: mon warn on no sortbitwise = false 2020-04-10T07:04:24.206 INFO:teuthology.orchestra.run.smithi204.stdout: mon warn on osd down out interval zero = false 2020-04-10T07:04:24.206 INFO:teuthology.orchestra.run.smithi204.stdout: mon warn on too few osds = false 2020-04-10T07:04:24.206 INFO:teuthology.orchestra.run.smithi204.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-04-10T07:04:24.207 INFO:teuthology.orchestra.run.smithi204.stdout: mon_warn_on_pool_no_redundancy = false 2020-04-10T07:04:24.207 INFO:teuthology.orchestra.run.smithi204.stdout: 2020-04-10T07:04:24.207 INFO:teuthology.orchestra.run.smithi204.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-04-10T07:04:24.207 INFO:teuthology.orchestra.run.smithi204.stdout: 2020-04-10T07:04:24.207 INFO:teuthology.orchestra.run.smithi204.stdout: osd default data pool replay window = 5 2020-04-10T07:04:24.207 INFO:teuthology.orchestra.run.smithi204.stdout: 2020-04-10T07:04:24.207 INFO:teuthology.orchestra.run.smithi204.stdout: mon allow pool delete = true 2020-04-10T07:04:24.208 INFO:teuthology.orchestra.run.smithi204.stdout: 2020-04-10T07:04:24.208 INFO:teuthology.orchestra.run.smithi204.stdout: mon cluster log file level = debug 2020-04-10T07:04:24.208 INFO:teuthology.orchestra.run.smithi204.stdout: debug asserts on shutdown = true 2020-04-10T07:04:24.208 INFO:teuthology.orchestra.run.smithi204.stdout: mon host = 172.21.15.204 2020-04-10T07:04:24.208 INFO:teuthology.orchestra.run.smithi204.stdout: ms inject socket failures = 5000 2020-04-10T07:04:24.208 INFO:teuthology.orchestra.run.smithi204.stdout: mon client directed command retry = 5 2020-04-10T07:04:24.208 INFO:teuthology.orchestra.run.smithi204.stdout: fsid = b4476b22-7943-481f-a1ec-34bc78d4adbc 2020-04-10T07:04:24.210 INFO:teuthology.orchestra.run.smithi204.stdout: 2020-04-10T07:04:24.210 INFO:teuthology.orchestra.run.smithi204.stdout:[osd] 2020-04-10T07:04:24.210 INFO:teuthology.orchestra.run.smithi204.stdout: osd journal size = 100 2020-04-10T07:04:24.210 INFO:teuthology.orchestra.run.smithi204.stdout: 2020-04-10T07:04:24.210 INFO:teuthology.orchestra.run.smithi204.stdout: osd scrub load threshold = 5.0 2020-04-10T07:04:24.210 INFO:teuthology.orchestra.run.smithi204.stdout: osd scrub max interval = 600 2020-04-10T07:04:24.210 INFO:teuthology.orchestra.run.smithi204.stdout: 2020-04-10T07:04:24.211 INFO:teuthology.orchestra.run.smithi204.stdout: osd recover clone overlap = true 2020-04-10T07:04:24.211 INFO:teuthology.orchestra.run.smithi204.stdout: osd recovery max chunk = 1048576 2020-04-10T07:04:24.211 INFO:teuthology.orchestra.run.smithi204.stdout: 2020-04-10T07:04:24.211 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug shutdown = true 2020-04-10T07:04:24.211 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug op order = true 2020-04-10T07:04:24.211 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug verify stray on activate = true 2020-04-10T07:04:24.211 INFO:teuthology.orchestra.run.smithi204.stdout: 2020-04-10T07:04:24.212 INFO:teuthology.orchestra.run.smithi204.stdout: osd open classes on start = true 2020-04-10T07:04:24.212 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug pg log writeout = true 2020-04-10T07:04:24.212 INFO:teuthology.orchestra.run.smithi204.stdout: 2020-04-10T07:04:24.212 INFO:teuthology.orchestra.run.smithi204.stdout: osd deep scrub update digest min age = 30 2020-04-10T07:04:24.212 INFO:teuthology.orchestra.run.smithi204.stdout: 2020-04-10T07:04:24.213 INFO:teuthology.orchestra.run.smithi204.stdout: osd map max advance = 10 2020-04-10T07:04:24.213 INFO:teuthology.orchestra.run.smithi204.stdout: 2020-04-10T07:04:24.213 INFO:teuthology.orchestra.run.smithi204.stdout: journal zero on create = true 2020-04-10T07:04:24.213 INFO:teuthology.orchestra.run.smithi204.stdout: 2020-04-10T07:04:24.213 INFO:teuthology.orchestra.run.smithi204.stdout: filestore ondisk finisher threads = 3 2020-04-10T07:04:24.213 INFO:teuthology.orchestra.run.smithi204.stdout: filestore apply finisher threads = 3 2020-04-10T07:04:24.213 INFO:teuthology.orchestra.run.smithi204.stdout: 2020-04-10T07:04:24.213 INFO:teuthology.orchestra.run.smithi204.stdout: bdev debug aio = true 2020-04-10T07:04:24.215 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug misdirected ops = true 2020-04-10T07:04:24.215 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd full ratio = 0.9 2020-04-10T07:04:24.215 INFO:teuthology.orchestra.run.smithi204.stdout: debug ms = 20 2020-04-10T07:04:24.215 INFO:teuthology.orchestra.run.smithi204.stdout: bluestore fsck on mount = True 2020-04-10T07:04:24.216 INFO:teuthology.orchestra.run.smithi204.stdout: debug osd = 25 2020-04-10T07:04:24.216 INFO:teuthology.orchestra.run.smithi204.stdout: debug bluestore = 20 2020-04-10T07:04:24.216 INFO:teuthology.orchestra.run.smithi204.stdout: debug bluefs = 20 2020-04-10T07:04:24.216 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd backfillfull_ratio = 0.85 2020-04-10T07:04:24.216 INFO:teuthology.orchestra.run.smithi204.stdout: debug rocksdb = 10 2020-04-10T07:04:24.217 INFO:teuthology.orchestra.run.smithi204.stdout: debug journal = 20 2020-04-10T07:04:24.217 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd nearfull ratio = 0.8 2020-04-10T07:04:24.217 INFO:teuthology.orchestra.run.smithi204.stdout: osd memory target = 1610612736 2020-04-10T07:04:24.217 INFO:teuthology.orchestra.run.smithi204.stdout: osd objectstore = bluestore 2020-04-10T07:04:24.217 INFO:teuthology.orchestra.run.smithi204.stdout: bdev async discard = True 2020-04-10T07:04:24.217 INFO:teuthology.orchestra.run.smithi204.stdout: bluestore block size = 96636764160 2020-04-10T07:04:24.217 INFO:teuthology.orchestra.run.smithi204.stdout: debug filestore = 20 2020-04-10T07:04:24.218 INFO:teuthology.orchestra.run.smithi204.stdout: bdev enable discard = True 2020-04-10T07:04:24.218 INFO:teuthology.orchestra.run.smithi204.stdout: osd failsafe full ratio = 0.95 2020-04-10T07:04:24.218 INFO:teuthology.orchestra.run.smithi204.stdout: 2020-04-10T07:04:24.218 INFO:teuthology.orchestra.run.smithi204.stdout:[mgr] 2020-04-10T07:04:24.218 INFO:teuthology.orchestra.run.smithi204.stdout: debug ms = 1 2020-04-10T07:04:24.218 INFO:teuthology.orchestra.run.smithi204.stdout: debug mgr = 20 2020-04-10T07:04:24.218 INFO:teuthology.orchestra.run.smithi204.stdout: debug mon = 20 2020-04-10T07:04:24.219 INFO:teuthology.orchestra.run.smithi204.stdout: debug auth = 20 2020-04-10T07:04:24.219 INFO:teuthology.orchestra.run.smithi204.stdout: mon reweight min pgs per osd = 4 2020-04-10T07:04:24.219 INFO:teuthology.orchestra.run.smithi204.stdout: mon reweight min bytes per osd = 10 2020-04-10T07:04:24.219 INFO:teuthology.orchestra.run.smithi204.stdout: mgr/telemetry/nag = false 2020-04-10T07:04:24.219 INFO:teuthology.orchestra.run.smithi204.stdout: 2020-04-10T07:04:24.220 INFO:teuthology.orchestra.run.smithi204.stdout:[mon] 2020-04-10T07:04:24.220 INFO:teuthology.orchestra.run.smithi204.stdout: debug ms = 1 2020-04-10T07:04:24.220 INFO:teuthology.orchestra.run.smithi204.stdout: debug mon = 20 2020-04-10T07:04:24.220 INFO:teuthology.orchestra.run.smithi204.stdout: debug paxos = 20 2020-04-10T07:04:24.220 INFO:teuthology.orchestra.run.smithi204.stdout: debug auth = 20 2020-04-10T07:04:24.220 INFO:teuthology.orchestra.run.smithi204.stdout: mon data avail warn = 5 2020-04-10T07:04:24.221 INFO:teuthology.orchestra.run.smithi204.stdout: mon mgr mkfs grace = 240 2020-04-10T07:04:24.221 INFO:teuthology.orchestra.run.smithi204.stdout: mon reweight min pgs per osd = 4 2020-04-10T07:04:24.221 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd reporter subtree level = osd 2020-04-10T07:04:24.221 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd prime pg temp = true 2020-04-10T07:04:24.221 INFO:teuthology.orchestra.run.smithi204.stdout: mon reweight min bytes per osd = 10 2020-04-10T07:04:24.221 INFO:teuthology.orchestra.run.smithi204.stdout: 2020-04-10T07:04:24.221 INFO:teuthology.orchestra.run.smithi204.stdout:[client] 2020-04-10T07:04:24.222 INFO:teuthology.orchestra.run.smithi204.stdout: rgw cache enabled = true 2020-04-10T07:04:24.222 INFO:teuthology.orchestra.run.smithi204.stdout: rgw enable ops log = true 2020-04-10T07:04:24.222 INFO:teuthology.orchestra.run.smithi204.stdout: rgw enable usage log = true 2020-04-10T07:04:24.222 INFO:teuthology.orchestra.run.smithi204.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-04-10T07:04:24.222 INFO:teuthology.orchestra.run.smithi204.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-04-10T07:04:24.222 INFO:teuthology.orchestra.run.smithi204.stdout: debug rbd = 20 2020-04-10T07:04:24.222 INFO:teuthology.orchestra.run.smithi204.stdout: rbd default features = 125 2020-04-10T07:04:24.223 INFO:teuthology.orchestra.run.smithi204.stdout: debug rbd_mirror = 15 2020-04-10T07:04:24.223 INFO:teuthology.orchestra.run.smithi204.stdout: log to stderr = False 2020-04-10T07:04:24.223 INFO:teuthology.orchestra.run.smithi204.stdout:[mon.a] 2020-04-10T07:04:24.223 INFO:teuthology.orchestra.run.smithi204.stdout:[client.mirror.5] 2020-04-10T07:04:24.223 INFO:teuthology.orchestra.run.smithi204.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2020-04-10T07:04:24.223 INFO:teuthology.orchestra.run.smithi204.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2020-04-10T07:04:24.223 INFO:teuthology.orchestra.run.smithi204.stdout:[client.mirror.4] 2020-04-10T07:04:24.223 INFO:teuthology.orchestra.run.smithi204.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2020-04-10T07:04:24.224 INFO:teuthology.orchestra.run.smithi204.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2020-04-10T07:04:24.224 INFO:teuthology.orchestra.run.smithi204.stdout:[client.mirror.6] 2020-04-10T07:04:24.224 INFO:teuthology.orchestra.run.smithi204.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2020-04-10T07:04:24.224 INFO:teuthology.orchestra.run.smithi204.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2020-04-10T07:04:24.224 INFO:teuthology.orchestra.run.smithi204.stdout:[client.mirror.1] 2020-04-10T07:04:24.224 INFO:teuthology.orchestra.run.smithi204.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2020-04-10T07:04:24.224 INFO:teuthology.orchestra.run.smithi204.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2020-04-10T07:04:24.225 INFO:teuthology.orchestra.run.smithi204.stdout:[client.mirror.0] 2020-04-10T07:04:24.225 INFO:teuthology.orchestra.run.smithi204.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2020-04-10T07:04:24.225 INFO:teuthology.orchestra.run.smithi204.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2020-04-10T07:04:24.225 INFO:teuthology.orchestra.run.smithi204.stdout:[client.mirror.3] 2020-04-10T07:04:24.225 INFO:teuthology.orchestra.run.smithi204.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2020-04-10T07:04:24.225 INFO:teuthology.orchestra.run.smithi204.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2020-04-10T07:04:24.225 INFO:teuthology.orchestra.run.smithi204.stdout:[client.mirror.2] 2020-04-10T07:04:24.226 INFO:teuthology.orchestra.run.smithi204.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2020-04-10T07:04:24.226 INFO:teuthology.orchestra.run.smithi204.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2020-04-10T07:04:24.227 INFO:tasks.ceph:Creating admin key on cluster2.mon.a... 2020-04-10T07:04:24.227 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:24.275 INFO:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/cluster2.keyring 2020-04-10T07:04:24.362 INFO:tasks.ceph:Copying monmap to all nodes... 2020-04-10T07:04:24.414 DEBUG:teuthology.orchestra.remote:smithi204:/etc/ceph/cluster2.keyring is 206B 2020-04-10T07:04:24.432 DEBUG:teuthology.orchestra.remote:smithi204:/home/ubuntu/cephtest/cluster2.monmap is 179B 2020-04-10T07:04:24.441 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi101.front.sepia.ceph.com 2020-04-10T07:04:24.441 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:24.446 INFO:teuthology.orchestra.run.smithi101:> sudo sh -c 'cat > /etc/ceph/cluster2.keyring' && sudo chmod 0644 /etc/ceph/cluster2.keyring 2020-04-10T07:04:24.544 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:24.593 INFO:teuthology.orchestra.run.smithi101:> cat > /home/ubuntu/cephtest/cluster2.monmap 2020-04-10T07:04:24.682 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi204.front.sepia.ceph.com 2020-04-10T07:04:24.682 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:24.687 INFO:teuthology.orchestra.run.smithi204:> sudo sh -c 'cat > /etc/ceph/cluster2.keyring' && sudo chmod 0644 /etc/ceph/cluster2.keyring 2020-04-10T07:04:24.789 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:24.835 INFO:teuthology.orchestra.run.smithi204:> cat > /home/ubuntu/cephtest/cluster2.monmap 2020-04-10T07:04:24.923 INFO:tasks.ceph:Setting up mon nodes... 2020-04-10T07:04:24.924 INFO:tasks.ceph:Setting up mgr nodes... 2020-04-10T07:04:24.924 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:24.929 INFO:teuthology.orchestra.run.smithi101:> sudo mkdir -p /var/lib/ceph/mgr/cluster2-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/cluster2-x/keyring 2020-04-10T07:04:25.020 INFO:teuthology.orchestra.run.smithi101.stdout:creating /var/lib/ceph/mgr/cluster2-x/keyring 2020-04-10T07:04:25.022 INFO:tasks.ceph:Setting up mds nodes... 2020-04-10T07:04:25.022 INFO:tasks.ceph_client:Setting up client nodes... 2020-04-10T07:04:25.023 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:25.070 INFO:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/cluster2.client.0.keyring && sudo chmod 0644 /etc/ceph/cluster2.client.0.keyring 2020-04-10T07:04:25.144 INFO:teuthology.orchestra.run.smithi101.stdout:creating /etc/ceph/cluster2.client.0.keyring 2020-04-10T07:04:25.157 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:25.162 INFO:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror /etc/ceph/cluster2.client.mirror.keyring && sudo chmod 0644 /etc/ceph/cluster2.client.mirror.keyring 2020-04-10T07:04:25.238 INFO:teuthology.orchestra.run.smithi204.stdout:creating /etc/ceph/cluster2.client.mirror.keyring 2020-04-10T07:04:25.250 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:25.299 INFO:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.0 /etc/ceph/cluster2.client.mirror.0.keyring && sudo chmod 0644 /etc/ceph/cluster2.client.mirror.0.keyring 2020-04-10T07:04:25.374 INFO:teuthology.orchestra.run.smithi204.stdout:creating /etc/ceph/cluster2.client.mirror.0.keyring 2020-04-10T07:04:25.386 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:25.435 INFO:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.1 /etc/ceph/cluster2.client.mirror.1.keyring && sudo chmod 0644 /etc/ceph/cluster2.client.mirror.1.keyring 2020-04-10T07:04:25.510 INFO:teuthology.orchestra.run.smithi204.stdout:creating /etc/ceph/cluster2.client.mirror.1.keyring 2020-04-10T07:04:25.521 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:25.567 INFO:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.2 /etc/ceph/cluster2.client.mirror.2.keyring && sudo chmod 0644 /etc/ceph/cluster2.client.mirror.2.keyring 2020-04-10T07:04:25.642 INFO:teuthology.orchestra.run.smithi204.stdout:creating /etc/ceph/cluster2.client.mirror.2.keyring 2020-04-10T07:04:25.654 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:25.703 INFO:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.3 /etc/ceph/cluster2.client.mirror.3.keyring && sudo chmod 0644 /etc/ceph/cluster2.client.mirror.3.keyring 2020-04-10T07:04:25.780 INFO:teuthology.orchestra.run.smithi204.stdout:creating /etc/ceph/cluster2.client.mirror.3.keyring 2020-04-10T07:04:25.790 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:25.839 INFO:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.4 /etc/ceph/cluster2.client.mirror.4.keyring && sudo chmod 0644 /etc/ceph/cluster2.client.mirror.4.keyring 2020-04-10T07:04:25.914 INFO:teuthology.orchestra.run.smithi204.stdout:creating /etc/ceph/cluster2.client.mirror.4.keyring 2020-04-10T07:04:25.925 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:25.971 INFO:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.5 /etc/ceph/cluster2.client.mirror.5.keyring && sudo chmod 0644 /etc/ceph/cluster2.client.mirror.5.keyring 2020-04-10T07:04:26.046 INFO:teuthology.orchestra.run.smithi204.stdout:creating /etc/ceph/cluster2.client.mirror.5.keyring 2020-04-10T07:04:26.059 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:26.108 INFO:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.6 /etc/ceph/cluster2.client.mirror.6.keyring && sudo chmod 0644 /etc/ceph/cluster2.client.mirror.6.keyring 2020-04-10T07:04:26.182 INFO:teuthology.orchestra.run.smithi204.stdout:creating /etc/ceph/cluster2.client.mirror.6.keyring 2020-04-10T07:04:26.194 INFO:tasks.ceph:Running mkfs on osd nodes... 2020-04-10T07:04:26.194 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi101.front.sepia.ceph.com'): {'cluster1.osd.2': '/dev/vg_nvme/lv_2', 'cluster1.osd.1': '/dev/vg_nvme/lv_3', 'cluster1.osd.0': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi204.front.sepia.ceph.com'): {'cluster2.osd.0': '/dev/vg_nvme/lv_4', 'cluster2.osd.1': '/dev/vg_nvme/lv_3', 'cluster2.osd.2': '/dev/vg_nvme/lv_2'}} 2020-04-10T07:04:26.194 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:26.239 INFO:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/osd/cluster2-0 2020-04-10T07:04:26.296 INFO:tasks.ceph:roles_to_devs: {'cluster2.osd.0': '/dev/vg_nvme/lv_4', 'cluster2.osd.1': '/dev/vg_nvme/lv_3', 'cluster2.osd.2': '/dev/vg_nvme/lv_2'} 2020-04-10T07:04:26.297 INFO:tasks.ceph:role: cluster2.osd.0 2020-04-10T07:04:26.297 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi204.front.sepia.ceph.com 2020-04-10T07:04:26.297 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:26.343 INFO:teuthology.orchestra.run.smithi204:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2020-04-10T07:04:26.855 INFO:teuthology.orchestra.run.smithi204.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2020-04-10T07:04:26.856 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 attr=2, projid32bit=1 2020-04-10T07:04:26.856 INFO:teuthology.orchestra.run.smithi204.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2020-04-10T07:04:26.856 INFO:teuthology.orchestra.run.smithi204.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-04-10T07:04:26.856 INFO:teuthology.orchestra.run.smithi204.stdout: = sunit=0 swidth=0 blks 2020-04-10T07:04:26.856 INFO:teuthology.orchestra.run.smithi204.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2020-04-10T07:04:26.856 INFO:teuthology.orchestra.run.smithi204.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-04-10T07:04:26.856 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-04-10T07:04:26.857 INFO:teuthology.orchestra.run.smithi204.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-04-10T07:04:26.857 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi204.front.sepia.ceph.com -o noatime 2020-04-10T07:04:26.858 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:26.863 INFO:teuthology.orchestra.run.smithi204:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/cluster2-0 2020-04-10T07:04:26.997 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:27.003 INFO:teuthology.orchestra.run.smithi204:> sudo /sbin/restorecon /var/lib/ceph/osd/cluster2-0 2020-04-10T07:04:27.057 INFO:teuthology.orchestra.run.smithi204.stderr:sudo: /sbin/restorecon: command not found 2020-04-10T07:04:27.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-04-10T07:04:27.058 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:27.107 INFO:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/osd/cluster2-1 2020-04-10T07:04:27.164 INFO:tasks.ceph:roles_to_devs: {'cluster2.osd.0': '/dev/vg_nvme/lv_4', 'cluster2.osd.1': '/dev/vg_nvme/lv_3', 'cluster2.osd.2': '/dev/vg_nvme/lv_2'} 2020-04-10T07:04:27.164 INFO:tasks.ceph:role: cluster2.osd.1 2020-04-10T07:04:27.165 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi204.front.sepia.ceph.com 2020-04-10T07:04:27.165 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:27.211 INFO:teuthology.orchestra.run.smithi204:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2020-04-10T07:04:27.708 INFO:teuthology.orchestra.run.smithi204.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2020-04-10T07:04:27.708 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 attr=2, projid32bit=1 2020-04-10T07:04:27.708 INFO:teuthology.orchestra.run.smithi204.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2020-04-10T07:04:27.708 INFO:teuthology.orchestra.run.smithi204.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-04-10T07:04:27.709 INFO:teuthology.orchestra.run.smithi204.stdout: = sunit=0 swidth=0 blks 2020-04-10T07:04:27.709 INFO:teuthology.orchestra.run.smithi204.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2020-04-10T07:04:27.709 INFO:teuthology.orchestra.run.smithi204.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-04-10T07:04:27.709 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-04-10T07:04:27.709 INFO:teuthology.orchestra.run.smithi204.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-04-10T07:04:27.710 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi204.front.sepia.ceph.com -o noatime 2020-04-10T07:04:27.710 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:27.715 INFO:teuthology.orchestra.run.smithi204:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/cluster2-1 2020-04-10T07:04:27.809 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:27.814 INFO:teuthology.orchestra.run.smithi204:> sudo /sbin/restorecon /var/lib/ceph/osd/cluster2-1 2020-04-10T07:04:27.869 INFO:teuthology.orchestra.run.smithi204.stderr:sudo: /sbin/restorecon: command not found 2020-04-10T07:04:27.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-04-10T07:04:27.870 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:27.915 INFO:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/osd/cluster2-2 2020-04-10T07:04:27.972 INFO:tasks.ceph:roles_to_devs: {'cluster2.osd.0': '/dev/vg_nvme/lv_4', 'cluster2.osd.1': '/dev/vg_nvme/lv_3', 'cluster2.osd.2': '/dev/vg_nvme/lv_2'} 2020-04-10T07:04:27.973 INFO:tasks.ceph:role: cluster2.osd.2 2020-04-10T07:04:27.973 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi204.front.sepia.ceph.com 2020-04-10T07:04:27.973 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:28.019 INFO:teuthology.orchestra.run.smithi204:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2020-04-10T07:04:28.532 INFO:teuthology.orchestra.run.smithi204.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2020-04-10T07:04:28.532 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 attr=2, projid32bit=1 2020-04-10T07:04:28.532 INFO:teuthology.orchestra.run.smithi204.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2020-04-10T07:04:28.532 INFO:teuthology.orchestra.run.smithi204.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-04-10T07:04:28.532 INFO:teuthology.orchestra.run.smithi204.stdout: = sunit=0 swidth=0 blks 2020-04-10T07:04:28.532 INFO:teuthology.orchestra.run.smithi204.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2020-04-10T07:04:28.533 INFO:teuthology.orchestra.run.smithi204.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-04-10T07:04:28.533 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-04-10T07:04:28.533 INFO:teuthology.orchestra.run.smithi204.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-04-10T07:04:28.534 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi204.front.sepia.ceph.com -o noatime 2020-04-10T07:04:28.534 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:28.538 INFO:teuthology.orchestra.run.smithi204:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/cluster2-2 2020-04-10T07:04:28.657 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:28.662 INFO:teuthology.orchestra.run.smithi204:> sudo /sbin/restorecon /var/lib/ceph/osd/cluster2-2 2020-04-10T07:04:28.717 INFO:teuthology.orchestra.run.smithi204.stderr:sudo: /sbin/restorecon: command not found 2020-04-10T07:04:28.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-04-10T07:04:28.717 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:28.763 INFO:teuthology.orchestra.run.smithi204:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster cluster2 --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/cluster2.monmap 2020-04-10T07:04:28.866 INFO:teuthology.orchestra.run.smithi204.stderr:2020-04-10T07:04:28.863+0000 7f6349462d80 -1 auth: error reading file: /var/lib/ceph/osd/cluster2-0/keyring: can't open /var/lib/ceph/osd/cluster2-0/keyring: (2) No such file or directory 2020-04-10T07:04:28.869 INFO:teuthology.orchestra.run.smithi204.stderr:2020-04-10T07:04:28.867+0000 7f6349462d80 -1 created new key in keyring /var/lib/ceph/osd/cluster2-0/keyring 2020-04-10T07:04:28.869 INFO:teuthology.orchestra.run.smithi204.stderr:2020-04-10T07:04:28.867+0000 7f6349462d80 -1 bluestore(/var/lib/ceph/osd/cluster2-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster2-0/block: (2) No such file or directory 2020-04-10T07:04:28.870 INFO:teuthology.orchestra.run.smithi204.stderr:2020-04-10T07:04:28.867+0000 7f6349462d80 -1 bluestore(/var/lib/ceph/osd/cluster2-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster2-0/block: (2) No such file or directory 2020-04-10T07:04:28.870 INFO:teuthology.orchestra.run.smithi204.stderr:2020-04-10T07:04:28.867+0000 7f6349462d80 -1 bluestore(/var/lib/ceph/osd/cluster2-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster2-0/block: (2) No such file or directory 2020-04-10T07:04:28.879 INFO:teuthology.orchestra.run.smithi204.stderr:2020-04-10T07:04:28.879+0000 7f6349462d80 -1 bluestore(/var/lib/ceph/osd/cluster2-0) _read_fsid unparsable uuid 2020-04-10T07:04:31.318 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:31.323 INFO:teuthology.orchestra.run.smithi204:> sudo chown -R ceph:ceph /var/lib/ceph/osd/cluster2-0 2020-04-10T07:04:31.381 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:31.427 INFO:teuthology.orchestra.run.smithi204:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster cluster2 --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/cluster2.monmap 2020-04-10T07:04:31.511 INFO:teuthology.orchestra.run.smithi204.stderr:2020-04-10T07:04:31.507+0000 7f55a6074d80 -1 auth: error reading file: /var/lib/ceph/osd/cluster2-1/keyring: can't open /var/lib/ceph/osd/cluster2-1/keyring: (2) No such file or directory 2020-04-10T07:04:31.511 INFO:teuthology.orchestra.run.smithi204.stderr:2020-04-10T07:04:31.511+0000 7f55a6074d80 -1 created new key in keyring /var/lib/ceph/osd/cluster2-1/keyring 2020-04-10T07:04:31.511 INFO:teuthology.orchestra.run.smithi204.stderr:2020-04-10T07:04:31.511+0000 7f55a6074d80 -1 bluestore(/var/lib/ceph/osd/cluster2-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster2-1/block: (2) No such file or directory 2020-04-10T07:04:31.511 INFO:teuthology.orchestra.run.smithi204.stderr:2020-04-10T07:04:31.511+0000 7f55a6074d80 -1 bluestore(/var/lib/ceph/osd/cluster2-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster2-1/block: (2) No such file or directory 2020-04-10T07:04:31.511 INFO:teuthology.orchestra.run.smithi204.stderr:2020-04-10T07:04:31.511+0000 7f55a6074d80 -1 bluestore(/var/lib/ceph/osd/cluster2-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster2-1/block: (2) No such file or directory 2020-04-10T07:04:31.512 INFO:teuthology.orchestra.run.smithi204.stderr:2020-04-10T07:04:31.511+0000 7f55a6074d80 -1 bluestore(/var/lib/ceph/osd/cluster2-1) _read_fsid unparsable uuid 2020-04-10T07:04:33.945 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:33.995 INFO:teuthology.orchestra.run.smithi204:> sudo chown -R ceph:ceph /var/lib/ceph/osd/cluster2-1 2020-04-10T07:04:34.053 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:34.103 INFO:teuthology.orchestra.run.smithi204:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster cluster2 --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/cluster2.monmap 2020-04-10T07:04:34.183 INFO:teuthology.orchestra.run.smithi204.stderr:2020-04-10T07:04:34.179+0000 7f9dcd35cd80 -1 auth: error reading file: /var/lib/ceph/osd/cluster2-2/keyring: can't open /var/lib/ceph/osd/cluster2-2/keyring: (2) No such file or directory 2020-04-10T07:04:34.183 INFO:teuthology.orchestra.run.smithi204.stderr:2020-04-10T07:04:34.183+0000 7f9dcd35cd80 -1 created new key in keyring /var/lib/ceph/osd/cluster2-2/keyring 2020-04-10T07:04:34.183 INFO:teuthology.orchestra.run.smithi204.stderr:2020-04-10T07:04:34.183+0000 7f9dcd35cd80 -1 bluestore(/var/lib/ceph/osd/cluster2-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster2-2/block: (2) No such file or directory 2020-04-10T07:04:34.183 INFO:teuthology.orchestra.run.smithi204.stderr:2020-04-10T07:04:34.183+0000 7f9dcd35cd80 -1 bluestore(/var/lib/ceph/osd/cluster2-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster2-2/block: (2) No such file or directory 2020-04-10T07:04:34.183 INFO:teuthology.orchestra.run.smithi204.stderr:2020-04-10T07:04:34.183+0000 7f9dcd35cd80 -1 bluestore(/var/lib/ceph/osd/cluster2-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster2-2/block: (2) No such file or directory 2020-04-10T07:04:34.184 INFO:teuthology.orchestra.run.smithi204.stderr:2020-04-10T07:04:34.183+0000 7f9dcd35cd80 -1 bluestore(/var/lib/ceph/osd/cluster2-2) _read_fsid unparsable uuid 2020-04-10T07:04:36.633 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:36.683 INFO:teuthology.orchestra.run.smithi204:> sudo chown -R ceph:ceph /var/lib/ceph/osd/cluster2-2 2020-04-10T07:04:36.745 INFO:tasks.ceph:Reading keys from all nodes... 2020-04-10T07:04:36.745 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:36.751 INFO:teuthology.orchestra.run.smithi101:> mktemp 2020-04-10T07:04:36.798 INFO:teuthology.orchestra.run.smithi101.stdout:/tmp/tmp.aIk0cthaSP 2020-04-10T07:04:36.798 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:36.845 INFO:teuthology.orchestra.run.smithi101:> sudo cp /var/lib/ceph/mgr/cluster2-x/keyring /tmp/tmp.aIk0cthaSP 2020-04-10T07:04:36.904 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:36.949 INFO:teuthology.orchestra.run.smithi101:> sudo chmod 0666 /tmp/tmp.aIk0cthaSP 2020-04-10T07:04:37.056 DEBUG:teuthology.orchestra.remote:smithi101:/tmp/tmp.aIk0cthaSP is 56B 2020-04-10T07:04:37.069 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:37.075 INFO:teuthology.orchestra.run.smithi101:> rm -fr /tmp/tmp.aIk0cthaSP 2020-04-10T07:04:37.123 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:37.128 INFO:teuthology.orchestra.run.smithi204:> mktemp 2020-04-10T07:04:37.175 INFO:teuthology.orchestra.run.smithi204.stdout:/tmp/tmp.oW9ko4VzuA 2020-04-10T07:04:37.176 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:37.222 INFO:teuthology.orchestra.run.smithi204:> sudo cp /var/lib/ceph/osd/cluster2-0/keyring /tmp/tmp.oW9ko4VzuA 2020-04-10T07:04:37.281 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:37.328 INFO:teuthology.orchestra.run.smithi204:> sudo chmod 0666 /tmp/tmp.oW9ko4VzuA 2020-04-10T07:04:37.433 DEBUG:teuthology.orchestra.remote:smithi204:/tmp/tmp.oW9ko4VzuA is 56B 2020-04-10T07:04:37.446 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:37.451 INFO:teuthology.orchestra.run.smithi204:> rm -fr /tmp/tmp.oW9ko4VzuA 2020-04-10T07:04:37.501 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:37.546 INFO:teuthology.orchestra.run.smithi204:> mktemp 2020-04-10T07:04:37.595 INFO:teuthology.orchestra.run.smithi204.stdout:/tmp/tmp.CvAXQAjLSb 2020-04-10T07:04:37.596 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:37.642 INFO:teuthology.orchestra.run.smithi204:> sudo cp /var/lib/ceph/osd/cluster2-1/keyring /tmp/tmp.CvAXQAjLSb 2020-04-10T07:04:37.701 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:37.747 INFO:teuthology.orchestra.run.smithi204:> sudo chmod 0666 /tmp/tmp.CvAXQAjLSb 2020-04-10T07:04:37.853 DEBUG:teuthology.orchestra.remote:smithi204:/tmp/tmp.CvAXQAjLSb is 56B 2020-04-10T07:04:37.866 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:37.870 INFO:teuthology.orchestra.run.smithi204:> rm -fr /tmp/tmp.CvAXQAjLSb 2020-04-10T07:04:37.920 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:37.966 INFO:teuthology.orchestra.run.smithi204:> mktemp 2020-04-10T07:04:38.017 INFO:teuthology.orchestra.run.smithi204.stdout:/tmp/tmp.N2gC91hFIN 2020-04-10T07:04:38.017 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:38.062 INFO:teuthology.orchestra.run.smithi204:> sudo cp /var/lib/ceph/osd/cluster2-2/keyring /tmp/tmp.N2gC91hFIN 2020-04-10T07:04:38.121 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:38.169 INFO:teuthology.orchestra.run.smithi204:> sudo chmod 0666 /tmp/tmp.N2gC91hFIN 2020-04-10T07:04:38.277 DEBUG:teuthology.orchestra.remote:smithi204:/tmp/tmp.N2gC91hFIN is 56B 2020-04-10T07:04:38.288 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:38.293 INFO:teuthology.orchestra.run.smithi204:> rm -fr /tmp/tmp.N2gC91hFIN 2020-04-10T07:04:38.348 DEBUG:teuthology.orchestra.remote:smithi101:/etc/ceph/cluster2.client.0.keyring is 59B 2020-04-10T07:04:38.389 DEBUG:teuthology.orchestra.remote:smithi204:/etc/ceph/cluster2.client.mirror.keyring is 64B 2020-04-10T07:04:38.409 DEBUG:teuthology.orchestra.remote:smithi204:/etc/ceph/cluster2.client.mirror.0.keyring is 66B 2020-04-10T07:04:38.431 DEBUG:teuthology.orchestra.remote:smithi204:/etc/ceph/cluster2.client.mirror.1.keyring is 66B 2020-04-10T07:04:38.454 DEBUG:teuthology.orchestra.remote:smithi204:/etc/ceph/cluster2.client.mirror.2.keyring is 66B 2020-04-10T07:04:38.472 DEBUG:teuthology.orchestra.remote:smithi204:/etc/ceph/cluster2.client.mirror.3.keyring is 66B 2020-04-10T07:04:38.497 DEBUG:teuthology.orchestra.remote:smithi204:/etc/ceph/cluster2.client.mirror.4.keyring is 66B 2020-04-10T07:04:38.564 DEBUG:teuthology.orchestra.remote:smithi204:/etc/ceph/cluster2.client.mirror.5.keyring is 66B 2020-04-10T07:04:38.582 DEBUG:teuthology.orchestra.remote:smithi204:/etc/ceph/cluster2.client.mirror.6.keyring is 66B 2020-04-10T07:04:38.592 INFO:tasks.ceph:Adding keys to all mons... 2020-04-10T07:04:38.592 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:38.597 INFO:teuthology.orchestra.run.smithi204:> sudo tee -a /etc/ceph/cluster2.keyring 2020-04-10T07:04:38.654 INFO:teuthology.orchestra.run.smithi204.stdout:[mgr.x] 2020-04-10T07:04:38.654 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQB5GpBevY1aARAAAV2nkKG/DNJwXHCU0z4a9g== 2020-04-10T07:04:38.654 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.0] 2020-04-10T07:04:38.654 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQB8GpBe7fGwMxAABCrV98dDoun34jLZyVI/sg== 2020-04-10T07:04:38.655 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.1] 2020-04-10T07:04:38.655 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQB/GpBe7Vl9HhAAGn+LgGtJFSkLSO1v9IsGTw== 2020-04-10T07:04:38.655 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.2] 2020-04-10T07:04:38.655 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQCCGpBeUqXvChAAID74To8qlxKKFwLXdW4OZw== 2020-04-10T07:04:38.655 INFO:teuthology.orchestra.run.smithi204.stdout:[client.0] 2020-04-10T07:04:38.655 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQB5GpBeCDXKCBAA7LQw0ShwrbAruR9KJQzc6Q== 2020-04-10T07:04:38.655 INFO:teuthology.orchestra.run.smithi204.stdout:[client.mirror] 2020-04-10T07:04:38.656 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQB5GpBe0JtEDhAAPiwcPIWq4N0xtqK+LJ+HAw== 2020-04-10T07:04:38.656 INFO:teuthology.orchestra.run.smithi204.stdout:[client.mirror.0] 2020-04-10T07:04:38.656 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQB5GpBe6TphFhAAq+cudlVUi9G1Mk8IR5JPeg== 2020-04-10T07:04:38.656 INFO:teuthology.orchestra.run.smithi204.stdout:[client.mirror.1] 2020-04-10T07:04:38.656 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQB5GpBeo9J9HhAArsozQU5vb0I78UhyMlBDDg== 2020-04-10T07:04:38.656 INFO:teuthology.orchestra.run.smithi204.stdout:[client.mirror.2] 2020-04-10T07:04:38.656 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQB5GpBe9dZaJhAAmFBL4XcH3dEP+Ex0G5s3qw== 2020-04-10T07:04:38.657 INFO:teuthology.orchestra.run.smithi204.stdout:[client.mirror.3] 2020-04-10T07:04:38.657 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQB5GpBeNsR3LhAASD7MrvxKm4le9NywjvNvJw== 2020-04-10T07:04:38.657 INFO:teuthology.orchestra.run.smithi204.stdout:[client.mirror.4] 2020-04-10T07:04:38.657 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQB5GpBelGCRNhAANN5Ct7JDJOZG6Fp27Ib2nQ== 2020-04-10T07:04:38.657 INFO:teuthology.orchestra.run.smithi204.stdout:[client.mirror.5] 2020-04-10T07:04:38.657 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQB6GpBerk7VAhAAOBQlzvEQWC4N1BiW5dpqOQ== 2020-04-10T07:04:38.657 INFO:teuthology.orchestra.run.smithi204.stdout:[client.mirror.6] 2020-04-10T07:04:38.658 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQB6GpBeNUXvChAA09FEVgavhyWG8gF+GNyHPA== 2020-04-10T07:04:38.658 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:38.703 INFO:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=mgr.x --cap mds 'allow *' --cap osd 'allow *' --cap mon 'allow profile mgr' 2020-04-10T07:04:38.790 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:38.840 INFO:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=osd.0 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2020-04-10T07:04:38.924 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:38.972 INFO:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=osd.1 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2020-04-10T07:04:39.055 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:39.105 INFO:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=osd.2 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2020-04-10T07:04:39.188 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:39.236 INFO:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=client.0 --cap mds allow --cap osd 'allow rwx' --cap mgr 'allow r' --cap mon 'allow rw' 2020-04-10T07:04:39.323 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:39.371 INFO:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=client.mirror --cap mds allow --cap osd 'allow rwx' --cap mgr 'allow r' --cap mon 'allow rw' 2020-04-10T07:04:39.455 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:39.505 INFO:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=client.mirror.0 --cap mds allow --cap osd 'allow rwx' --cap mgr 'allow r' --cap mon 'allow rw' 2020-04-10T07:04:39.587 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:39.635 INFO:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=client.mirror.1 --cap mds allow --cap osd 'allow rwx' --cap mgr 'allow r' --cap mon 'allow rw' 2020-04-10T07:04:39.722 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:39.771 INFO:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=client.mirror.2 --cap mds allow --cap osd 'allow rwx' --cap mgr 'allow r' --cap mon 'allow rw' 2020-04-10T07:04:39.858 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:39.912 INFO:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=client.mirror.3 --cap mds allow --cap osd 'allow rwx' --cap mgr 'allow r' --cap mon 'allow rw' 2020-04-10T07:04:40.005 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:40.060 INFO:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=client.mirror.4 --cap mds allow --cap osd 'allow rwx' --cap mgr 'allow r' --cap mon 'allow rw' 2020-04-10T07:04:40.153 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:40.199 INFO:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=client.mirror.5 --cap mds allow --cap osd 'allow rwx' --cap mgr 'allow r' --cap mon 'allow rw' 2020-04-10T07:04:40.287 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:40.292 INFO:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=client.mirror.6 --cap mds allow --cap osd 'allow rwx' --cap mgr 'allow r' --cap mon 'allow rw' 2020-04-10T07:04:40.378 INFO:tasks.ceph:Running mkfs on mon nodes... 2020-04-10T07:04:40.379 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:40.427 INFO:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/mon/cluster2-a 2020-04-10T07:04:40.485 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:40.531 INFO:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster cluster2 --mkfs -i a --monmap /home/ubuntu/cephtest/cluster2.monmap --keyring /etc/ceph/cluster2.keyring 2020-04-10T07:04:40.650 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:40.656 INFO:teuthology.orchestra.run.smithi204:> sudo chown -R ceph:ceph /var/lib/ceph/mon/cluster2-a 2020-04-10T07:04:40.715 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:40.763 INFO:teuthology.orchestra.run.smithi204:> rm -- /home/ubuntu/cephtest/cluster2.monmap 2020-04-10T07:04:40.810 INFO:tasks.ceph:Starting mon daemons in cluster cluster2... 2020-04-10T07:04:40.811 INFO:tasks.ceph.cluster2.mon.a:Restarting daemon 2020-04-10T07:04:40.811 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:40.859 INFO:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster cluster2 -i a 2020-04-10T07:04:40.904 INFO:tasks.ceph.cluster2.mon.a:Started 2020-04-10T07:04:40.904 INFO:tasks.ceph:Starting mgr daemons in cluster cluster2... 2020-04-10T07:04:40.904 INFO:tasks.ceph.cluster2.mgr.x:Restarting daemon 2020-04-10T07:04:40.904 INFO:teuthology.orchestra.run.smithi101:> true 2020-04-10T07:04:40.910 INFO:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster cluster2 -i x 2020-04-10T07:04:40.954 INFO:tasks.ceph.cluster2.mgr.x:Started 2020-04-10T07:04:40.954 INFO:tasks.ceph:Setting crush tunables to default 2020-04-10T07:04:40.954 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:40.960 INFO:teuthology.orchestra.run.smithi204:> sudo ceph --cluster cluster2 osd crush tunables default 2020-04-10T07:04:41.284 INFO:teuthology.orchestra.run.smithi204.stderr:adjusted tunables profile to default 2020-04-10T07:04:41.294 INFO:tasks.ceph:Starting osd daemons in cluster cluster2... 2020-04-10T07:04:41.294 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:41.300 INFO:teuthology.orchestra.run.smithi204:> mktemp 2020-04-10T07:04:41.347 INFO:teuthology.orchestra.run.smithi204.stdout:/tmp/tmp.xtNpnnnviA 2020-04-10T07:04:41.348 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:41.394 INFO:teuthology.orchestra.run.smithi204:> sudo cp /var/lib/ceph/osd/cluster2-0/fsid /tmp/tmp.xtNpnnnviA 2020-04-10T07:04:41.453 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:41.500 INFO:teuthology.orchestra.run.smithi204:> sudo chmod 0666 /tmp/tmp.xtNpnnnviA 2020-04-10T07:04:41.609 DEBUG:teuthology.orchestra.remote:smithi204:/tmp/tmp.xtNpnnnviA is 37B 2020-04-10T07:04:41.620 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:41.625 INFO:teuthology.orchestra.run.smithi204:> rm -fr /tmp/tmp.xtNpnnnviA 2020-04-10T07:04:41.671 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:41.721 INFO:teuthology.orchestra.run.smithi204:> mktemp 2020-04-10T07:04:41.767 INFO:teuthology.orchestra.run.smithi204.stdout:/tmp/tmp.tWbOdiKXTt 2020-04-10T07:04:41.768 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:41.816 INFO:teuthology.orchestra.run.smithi204:> sudo cp /var/lib/ceph/osd/cluster2-1/fsid /tmp/tmp.tWbOdiKXTt 2020-04-10T07:04:41.872 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:41.925 INFO:teuthology.orchestra.run.smithi204:> sudo chmod 0666 /tmp/tmp.tWbOdiKXTt 2020-04-10T07:04:42.031 DEBUG:teuthology.orchestra.remote:smithi204:/tmp/tmp.tWbOdiKXTt is 37B 2020-04-10T07:04:42.041 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:42.045 INFO:teuthology.orchestra.run.smithi204:> rm -fr /tmp/tmp.tWbOdiKXTt 2020-04-10T07:04:42.095 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:42.142 INFO:teuthology.orchestra.run.smithi204:> mktemp 2020-04-10T07:04:42.191 INFO:teuthology.orchestra.run.smithi204.stdout:/tmp/tmp.jhLPayQSFt 2020-04-10T07:04:42.191 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:42.239 INFO:teuthology.orchestra.run.smithi204:> sudo cp /var/lib/ceph/osd/cluster2-2/fsid /tmp/tmp.jhLPayQSFt 2020-04-10T07:04:42.297 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:42.344 INFO:teuthology.orchestra.run.smithi204:> sudo chmod 0666 /tmp/tmp.jhLPayQSFt 2020-04-10T07:04:42.454 DEBUG:teuthology.orchestra.remote:smithi204:/tmp/tmp.jhLPayQSFt is 37B 2020-04-10T07:04:42.485 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:42.490 INFO:teuthology.orchestra.run.smithi204:> rm -fr /tmp/tmp.jhLPayQSFt 2020-04-10T07:04:42.539 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:42.586 INFO:teuthology.orchestra.run.smithi204:> sudo ceph --cluster cluster2 osd new b5153940-bfec-4327-bd6c-d384e65eb61f 0 2020-04-10T07:04:42.846 INFO:teuthology.orchestra.run.smithi204.stdout:0 2020-04-10T07:04:42.860 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:42.865 INFO:teuthology.orchestra.run.smithi204:> sudo ceph --cluster cluster2 osd new 029414ba-e331-47ff-b6c0-875139f3c352 1 2020-04-10T07:04:43.120 INFO:teuthology.orchestra.run.smithi204.stdout:1 2020-04-10T07:04:43.131 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:43.136 INFO:teuthology.orchestra.run.smithi204:> sudo ceph --cluster cluster2 osd new 316fbae1-abf1-404e-a6ef-fb2c241685d1 2 2020-04-10T07:04:43.387 INFO:teuthology.orchestra.run.smithi204.stdout:2 2020-04-10T07:04:43.399 INFO:tasks.ceph.cluster2.osd.0:Restarting daemon 2020-04-10T07:04:43.399 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:43.404 INFO:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster cluster2 -i 0 2020-04-10T07:04:43.447 INFO:tasks.ceph.cluster2.osd.0:Started 2020-04-10T07:04:43.448 INFO:tasks.ceph.cluster2.osd.1:Restarting daemon 2020-04-10T07:04:43.448 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:43.453 INFO:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster cluster2 -i 1 2020-04-10T07:04:43.496 INFO:tasks.ceph.cluster2.osd.1:Started 2020-04-10T07:04:43.496 INFO:tasks.ceph.cluster2.osd.2:Restarting daemon 2020-04-10T07:04:43.496 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:43.500 INFO:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster cluster2 -i 2 2020-04-10T07:04:43.549 INFO:tasks.ceph.cluster2.osd.2:Started 2020-04-10T07:04:43.549 INFO:tasks.ceph:Waiting for OSDs to come up 2020-04-10T07:04:43.852 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:43.857 INFO:teuthology.orchestra.run.smithi204:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster cluster2 osd dump --format=json 2020-04-10T07:04:44.087 INFO:tasks.ceph.cluster2.osd.0.smithi204.stderr:2020-04-10T07:04:44.087+0000 7f035761bd80 -1 Falling back to public interface 2020-04-10T07:04:44.106 INFO:teuthology.misc.health.smithi204.stdout: 2020-04-10T07:04:44.106 INFO:teuthology.misc.health.smithi204.stdout:{"epoch":5,"fsid":"b4476b22-7943-481f-a1ec-34bc78d4adbc","created":"2020-04-10T07:04:41.038193+0000","modified":"2020-04-10T07:04:43.384475+0000","last_up_change":"0.000000","last_in_change":"0.000000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[],"osds":[{"osd":0,"uuid":"b5153940-bfec-4327-bd6c-d384e65eb61f","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"029414ba-e331-47ff-b6c0-875139f3c352","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"316fbae1-abf1-404e-a6ef-fb2c241685d1","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-04-10T07:04:44.117 INFO:tasks.ceph.cluster2.osd.1.smithi204.stderr:2020-04-10T07:04:44.115+0000 7f5de1f84d80 -1 Falling back to public interface 2020-04-10T07:04:44.118 DEBUG:teuthology.misc:0 of 3 OSDs are up 2020-04-10T07:04:44.161 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T07:04:44.159+0000 7fefe985ed80 -1 Falling back to public interface 2020-04-10T07:04:44.965 INFO:tasks.ceph.cluster2.osd.0.smithi204.stderr:2020-04-10T07:04:44.963+0000 7f035761bd80 -1 osd.0 0 log_to_monitors {default=true} 2020-04-10T07:04:45.304 INFO:tasks.ceph.cluster2.osd.1.smithi204.stderr:2020-04-10T07:04:45.303+0000 7f5de1f84d80 -1 osd.1 0 log_to_monitors {default=true} 2020-04-10T07:04:45.338 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T07:04:45.335+0000 7fefe985ed80 -1 osd.2 0 log_to_monitors {default=true} 2020-04-10T07:04:46.799 INFO:tasks.ceph.cluster2.osd.0.smithi204.stderr:2020-04-10T07:04:46.799+0000 7f0340867700 -1 osd.0 0 waiting for initial osdmap 2020-04-10T07:04:46.800 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T07:04:46.799+0000 7fefd2aaa700 -1 osd.2 0 waiting for initial osdmap 2020-04-10T07:04:46.800 INFO:tasks.ceph.cluster2.osd.1.smithi204.stderr:2020-04-10T07:04:46.799+0000 7f5dcb1d0700 -1 osd.1 0 waiting for initial osdmap 2020-04-10T07:04:50.421 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:50.427 INFO:teuthology.orchestra.run.smithi204:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster cluster2 osd dump --format=json 2020-04-10T07:04:50.727 INFO:teuthology.misc.health.smithi204.stdout: 2020-04-10T07:04:50.728 INFO:teuthology.misc.health.smithi204.stdout:{"epoch":11,"fsid":"b4476b22-7943-481f-a1ec-34bc78d4adbc","created":"2020-04-10T07:04:41.038193+0000","modified":"2020-04-10T07:04:49.792701+0000","last_up_change":"2020-04-10T07:04:47.789699+0000","last_in_change":"2020-04-10T07:04:47.789699+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-04-10T07:04:44.030678+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}}],"osds":[{"osd":0,"uuid":"b5153940-bfec-4327-bd6c-d384e65eb61f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":12324},{"type":"v1","addr":"172.21.15.204:6801","nonce":12324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":12324},{"type":"v1","addr":"172.21.15.204:6803","nonce":12324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":12324},{"type":"v1","addr":"172.21.15.204:6807","nonce":12324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":12324},{"type":"v1","addr":"172.21.15.204:6805","nonce":12324}]},"public_addr":"172.21.15.204:6801/12324","cluster_addr":"172.21.15.204:6803/12324","heartbeat_back_addr":"172.21.15.204:6807/12324","heartbeat_front_addr":"172.21.15.204:6805/12324","state":["exists","up"]},{"osd":1,"uuid":"029414ba-e331-47ff-b6c0-875139f3c352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":12342},{"type":"v1","addr":"172.21.15.204:6809","nonce":12342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":12342},{"type":"v1","addr":"172.21.15.204:6811","nonce":12342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":12342},{"type":"v1","addr":"172.21.15.204:6815","nonce":12342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":12342},{"type":"v1","addr":"172.21.15.204:6813","nonce":12342}]},"public_addr":"172.21.15.204:6809/12342","cluster_addr":"172.21.15.204:6811/12342","heartbeat_back_addr":"172.21.15.204:6815/12342","heartbeat_front_addr":"172.21.15.204:6813/12342","state":["exists","up"]},{"osd":2,"uuid":"316fbae1-abf1-404e-a6ef-fb2c241685d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":12360},{"type":"v1","addr":"172.21.15.204:6817","nonce":12360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":12360},{"type":"v1","addr":"172.21.15.204:6819","nonce":12360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":12360},{"type":"v1","addr":"172.21.15.204:6823","nonce":12360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":12360},{"type":"v1","addr":"172.21.15.204:6821","nonce":12360}]},"public_addr":"172.21.15.204:6817/12360","cluster_addr":"172.21.15.204:6819/12360","heartbeat_back_addr":"172.21.15.204:6823/12360","heartbeat_front_addr":"172.21.15.204:6821/12360","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-04-10T07:04:45.977943+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-04-10T07:04:46.313577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-04-10T07:04:46.324207+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-04-10T07:04:50.739 DEBUG:teuthology.misc:3 of 3 OSDs are up 2020-04-10T07:04:50.739 INFO:tasks.ceph:Creating RBD pool 2020-04-10T07:04:50.739 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:50.745 INFO:teuthology.orchestra.run.smithi204:> sudo ceph --cluster cluster2 osd pool create rbd 8 2020-04-10T07:04:52.019 INFO:teuthology.orchestra.run.smithi204.stderr:pool 'rbd' created 2020-04-10T07:04:52.031 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:52.036 INFO:teuthology.orchestra.run.smithi204:> sudo ceph --cluster cluster2 osd pool application enable rbd rbd --yes-i-really-mean-it 2020-04-10T07:04:53.023 INFO:teuthology.orchestra.run.smithi204.stderr:enabled application 'rbd' on pool 'rbd' 2020-04-10T07:04:53.036 INFO:tasks.ceph:Starting mds daemons in cluster cluster2... 2020-04-10T07:04:53.036 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:53.037 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2020-04-10T07:04:53.041 INFO:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 --log-early osd dump --format=json 2020-04-10T07:04:53.355 INFO:teuthology.orchestra.run.smithi204.stdout: 2020-04-10T07:04:53.356 INFO:teuthology.orchestra.run.smithi204.stdout:{"epoch":13,"fsid":"b4476b22-7943-481f-a1ec-34bc78d4adbc","created":"2020-04-10T07:04:41.038193+0000","modified":"2020-04-10T07:04:53.007750+0000","last_up_change":"2020-04-10T07:04:47.789699+0000","last_in_change":"2020-04-10T07:04:47.789699+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-04-10T07:04:44.030678+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-04-10T07:04:51.088483+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"b5153940-bfec-4327-bd6c-d384e65eb61f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":12324},{"type":"v1","addr":"172.21.15.204:6801","nonce":12324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":12324},{"type":"v1","addr":"172.21.15.204:6803","nonce":12324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":12324},{"type":"v1","addr":"172.21.15.204:6807","nonce":12324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":12324},{"type":"v1","addr":"172.21.15.204:6805","nonce":12324}]},"public_addr":"172.21.15.204:6801/12324","cluster_addr":"172.21.15.204:6803/12324","heartbeat_back_addr":"172.21.15.204:6807/12324","heartbeat_front_addr":"172.21.15.204:6805/12324","state":["exists","up"]},{"osd":1,"uuid":"029414ba-e331-47ff-b6c0-875139f3c352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":12342},{"type":"v1","addr":"172.21.15.204:6809","nonce":12342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":12342},{"type":"v1","addr":"172.21.15.204:6811","nonce":12342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":12342},{"type":"v1","addr":"172.21.15.204:6815","nonce":12342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":12342},{"type":"v1","addr":"172.21.15.204:6813","nonce":12342}]},"public_addr":"172.21.15.204:6809/12342","cluster_addr":"172.21.15.204:6811/12342","heartbeat_back_addr":"172.21.15.204:6815/12342","heartbeat_front_addr":"172.21.15.204:6813/12342","state":["exists","up"]},{"osd":2,"uuid":"316fbae1-abf1-404e-a6ef-fb2c241685d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":12360},{"type":"v1","addr":"172.21.15.204:6817","nonce":12360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":12360},{"type":"v1","addr":"172.21.15.204:6819","nonce":12360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":12360},{"type":"v1","addr":"172.21.15.204:6823","nonce":12360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":12360},{"type":"v1","addr":"172.21.15.204:6821","nonce":12360}]},"public_addr":"172.21.15.204:6817/12360","cluster_addr":"172.21.15.204:6819/12360","heartbeat_back_addr":"172.21.15.204:6823/12360","heartbeat_front_addr":"172.21.15.204:6821/12360","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-04-10T07:04:45.977943+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-04-10T07:04:46.313577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-04-10T07:04:46.324207+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-04-10T07:04:53.369 INFO:tasks.ceph.ceph_manager.cluster2:[{u'cache_target_full_ratio_micro': 800000, u'fast_read': False, u'stripe_width': 0, u'flags_names': u'hashpspool', u'tier_of': -1, u'hit_set_grade_decay_rate': 0, u'pg_placement_num': 1, u'use_gmt_hitset': True, u'cache_target_dirty_ratio_micro': 400000, u'create_time': u'2020-04-10T07:04:44.030678+0000', u'snap_epoch': 0, u'erasure_code_profile': u'', u'pg_autoscale_mode': u'off', u'snap_seq': 0, u'expected_num_objects': 0, u'size': 2, u'pg_num_pending': 1, u'auid': 0, u'cache_min_flush_age': 0, u'hit_set_period': 0, u'min_read_recency_for_promote': 0, u'target_max_objects': 0, u'pg_placement_num_target': 1, u'pg_num': 1, u'type': 1, u'grade_table': [], u'pool_name': u'device_health_metrics', u'cache_min_evict_age': 0, u'snap_mode': u'selfmanaged', u'pg_num_target': 1, u'cache_mode': u'none', u'min_size': 1, u'cache_target_dirty_high_ratio_micro': 600000, u'object_hash': 2, u'last_pg_merge_meta': {u'ready_epoch': 0, u'source_version': u"0'0", u'source_pgid': u'0.0', u'last_epoch_clean': 0, u'target_version': u"0'0", u'last_epoch_started': 0}, u'write_tier': -1, u'last_force_op_resend_preluminous': u'0', u'pool': 1, u'removed_snaps': u'[]', u'crush_rule': 0, u'tiers': [], u'hit_set_params': {u'type': u'none'}, u'last_force_op_resend': u'0', u'pool_snaps': [], u'quota_max_objects': 0, u'last_force_op_resend_prenautilus': u'0', u'application_metadata': {u'mgr_devicehealth': {}}, u'options': {u'pg_num_min': 1}, u'hit_set_count': 0, u'flags': 1, u'target_max_bytes': 0, u'quota_max_bytes': 0, u'hit_set_search_last_n': 0, u'last_change': u'11', u'min_write_recency_for_promote': 0, u'read_tier': -1}, {u'cache_target_full_ratio_micro': 800000, u'fast_read': False, u'stripe_width': 0, u'flags_names': u'hashpspool,creating', u'tier_of': -1, u'hit_set_grade_decay_rate': 0, u'pg_placement_num': 8, u'use_gmt_hitset': True, u'cache_target_dirty_ratio_micro': 400000, u'create_time': u'2020-04-10T07:04:51.088483+0000', u'snap_epoch': 0, u'erasure_code_profile': u'', u'pg_autoscale_mode': u'off', u'snap_seq': 0, u'expected_num_objects': 0, u'size': 2, u'pg_num_pending': 8, u'auid': 0, u'cache_min_flush_age': 0, u'hit_set_period': 0, u'min_read_recency_for_promote': 0, u'target_max_objects': 0, u'pg_placement_num_target': 8, u'pg_num': 8, u'type': 1, u'grade_table': [], u'pool_name': u'rbd', u'cache_min_evict_age': 0, u'snap_mode': u'selfmanaged', u'pg_num_target': 8, u'cache_mode': u'none', u'min_size': 1, u'cache_target_dirty_high_ratio_micro': 600000, u'object_hash': 2, u'last_pg_merge_meta': {u'ready_epoch': 0, u'source_version': u"0'0", u'source_pgid': u'0.0', u'last_epoch_clean': 0, u'target_version': u"0'0", u'last_epoch_started': 0}, u'write_tier': -1, u'last_force_op_resend_preluminous': u'0', u'pool': 2, u'removed_snaps': u'[]', u'crush_rule': 0, u'tiers': [], u'hit_set_params': {u'type': u'none'}, u'last_force_op_resend': u'0', u'pool_snaps': [], u'quota_max_objects': 0, u'last_force_op_resend_prenautilus': u'0', u'application_metadata': {u'rbd': {}}, u'options': {}, u'hit_set_count': 0, u'flags': 32769, u'target_max_bytes': 0, u'quota_max_bytes': 0, u'hit_set_search_last_n': 0, u'last_change': u'13', u'min_write_recency_for_promote': 0, u'read_tier': -1}] 2020-04-10T07:04:53.373 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:53.379 INFO:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 --log-early osd pool get device_health_metrics pg_num 2020-04-10T07:04:53.685 INFO:teuthology.orchestra.run.smithi204.stdout:pg_num: 1 2020-04-10T07:04:53.698 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:53.702 INFO:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 --log-early osd pool get rbd pg_num 2020-04-10T07:04:54.009 INFO:teuthology.orchestra.run.smithi204.stdout:pg_num: 8 2020-04-10T07:04:54.022 INFO:tasks.ceph:Waiting until cluster2 daemons up and pgs clean... 2020-04-10T07:04:54.022 INFO:tasks.ceph.ceph_manager.cluster2:waiting for mgr available 2020-04-10T07:04:54.022 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:54.028 INFO:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 --log-early mgr dump --format=json 2020-04-10T07:04:54.368 INFO:teuthology.orchestra.run.smithi204.stdout: 2020-04-10T07:04:54.382 INFO:teuthology.orchestra.run.smithi204.stdout:{"epoch":4,"active_gid":4098,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6826","nonce":13505},{"type":"v1","addr":"172.21.15.101:6827","nonce":13505}]},"active_addr":"172.21.15.101:6827/13505","active_change":"2020-04-10T07:04:43.772745+0000","active_mgr_features":4540138292836696063,"available":true,"standbys":[],"modules":["iostat","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"none","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading remoto library:No module named 'remoto'","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"docker.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_nfs":{"name":"FEATURE_TOGGLE_nfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_cloud","can_run":true,"error_string":"","module_options":{"diskprediction_cert_context":{"name":"diskprediction_cert_context","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_default_authority":{"name":"diskprediction_default_authority","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_password":{"name":"diskprediction_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_port":{"name":"diskprediction_port","type":"str","level":"advanced","flags":0,"default_value":"31400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_retrieve_prediction_interval":{"name":"diskprediction_retrieve_prediction_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_server":{"name":"diskprediction_server","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_ssl_target_name_override":{"name":"diskprediction_ssl_target_name_override","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_metrics_interval":{"name":"diskprediction_upload_metrics_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_smart_interval":{"name":"diskprediction_upload_smart_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_user":{"name":"diskprediction_user","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"None","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator","osd_support","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.101:0","nonce":4221484477}]},{"addrvec":[{"type":"v2","addr":"172.21.15.101:0","nonce":1486688956}]},{"addrvec":[{"type":"v2","addr":"172.21.15.101:0","nonce":3427392092}]}]}} 2020-04-10T07:04:54.391 INFO:tasks.ceph.ceph_manager.cluster2:mgr available! 2020-04-10T07:04:54.391 INFO:tasks.ceph.ceph_manager.cluster2:waiting for all up 2020-04-10T07:04:54.391 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:54.398 INFO:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 --log-early osd dump --format=json 2020-04-10T07:04:54.703 INFO:teuthology.orchestra.run.smithi204.stdout: 2020-04-10T07:04:54.704 INFO:teuthology.orchestra.run.smithi204.stdout:{"epoch":14,"fsid":"b4476b22-7943-481f-a1ec-34bc78d4adbc","created":"2020-04-10T07:04:41.038193+0000","modified":"2020-04-10T07:04:54.017839+0000","last_up_change":"2020-04-10T07:04:47.789699+0000","last_in_change":"2020-04-10T07:04:47.789699+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-04-10T07:04:44.030678+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-04-10T07:04:51.088483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"b5153940-bfec-4327-bd6c-d384e65eb61f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":12324},{"type":"v1","addr":"172.21.15.204:6801","nonce":12324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":12324},{"type":"v1","addr":"172.21.15.204:6803","nonce":12324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":12324},{"type":"v1","addr":"172.21.15.204:6807","nonce":12324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":12324},{"type":"v1","addr":"172.21.15.204:6805","nonce":12324}]},"public_addr":"172.21.15.204:6801/12324","cluster_addr":"172.21.15.204:6803/12324","heartbeat_back_addr":"172.21.15.204:6807/12324","heartbeat_front_addr":"172.21.15.204:6805/12324","state":["exists","up"]},{"osd":1,"uuid":"029414ba-e331-47ff-b6c0-875139f3c352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":12342},{"type":"v1","addr":"172.21.15.204:6809","nonce":12342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":12342},{"type":"v1","addr":"172.21.15.204:6811","nonce":12342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":12342},{"type":"v1","addr":"172.21.15.204:6815","nonce":12342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":12342},{"type":"v1","addr":"172.21.15.204:6813","nonce":12342}]},"public_addr":"172.21.15.204:6809/12342","cluster_addr":"172.21.15.204:6811/12342","heartbeat_back_addr":"172.21.15.204:6815/12342","heartbeat_front_addr":"172.21.15.204:6813/12342","state":["exists","up"]},{"osd":2,"uuid":"316fbae1-abf1-404e-a6ef-fb2c241685d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":12360},{"type":"v1","addr":"172.21.15.204:6817","nonce":12360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":12360},{"type":"v1","addr":"172.21.15.204:6819","nonce":12360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":12360},{"type":"v1","addr":"172.21.15.204:6823","nonce":12360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":12360},{"type":"v1","addr":"172.21.15.204:6821","nonce":12360}]},"public_addr":"172.21.15.204:6817/12360","cluster_addr":"172.21.15.204:6819/12360","heartbeat_back_addr":"172.21.15.204:6823/12360","heartbeat_front_addr":"172.21.15.204:6821/12360","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-04-10T07:04:45.977943+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-04-10T07:04:46.313577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-04-10T07:04:46.324207+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-04-10T07:04:54.716 INFO:tasks.ceph.ceph_manager.cluster2:all up! 2020-04-10T07:04:54.716 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:54.721 INFO:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 --log-early osd dump --format=json 2020-04-10T07:04:55.022 INFO:teuthology.orchestra.run.smithi204.stdout: 2020-04-10T07:04:55.022 INFO:teuthology.orchestra.run.smithi204.stdout:{"epoch":14,"fsid":"b4476b22-7943-481f-a1ec-34bc78d4adbc","created":"2020-04-10T07:04:41.038193+0000","modified":"2020-04-10T07:04:54.017839+0000","last_up_change":"2020-04-10T07:04:47.789699+0000","last_in_change":"2020-04-10T07:04:47.789699+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-04-10T07:04:44.030678+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-04-10T07:04:51.088483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"b5153940-bfec-4327-bd6c-d384e65eb61f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":12324},{"type":"v1","addr":"172.21.15.204:6801","nonce":12324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":12324},{"type":"v1","addr":"172.21.15.204:6803","nonce":12324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":12324},{"type":"v1","addr":"172.21.15.204:6807","nonce":12324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":12324},{"type":"v1","addr":"172.21.15.204:6805","nonce":12324}]},"public_addr":"172.21.15.204:6801/12324","cluster_addr":"172.21.15.204:6803/12324","heartbeat_back_addr":"172.21.15.204:6807/12324","heartbeat_front_addr":"172.21.15.204:6805/12324","state":["exists","up"]},{"osd":1,"uuid":"029414ba-e331-47ff-b6c0-875139f3c352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":12342},{"type":"v1","addr":"172.21.15.204:6809","nonce":12342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":12342},{"type":"v1","addr":"172.21.15.204:6811","nonce":12342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":12342},{"type":"v1","addr":"172.21.15.204:6815","nonce":12342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":12342},{"type":"v1","addr":"172.21.15.204:6813","nonce":12342}]},"public_addr":"172.21.15.204:6809/12342","cluster_addr":"172.21.15.204:6811/12342","heartbeat_back_addr":"172.21.15.204:6815/12342","heartbeat_front_addr":"172.21.15.204:6813/12342","state":["exists","up"]},{"osd":2,"uuid":"316fbae1-abf1-404e-a6ef-fb2c241685d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":12360},{"type":"v1","addr":"172.21.15.204:6817","nonce":12360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":12360},{"type":"v1","addr":"172.21.15.204:6819","nonce":12360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":12360},{"type":"v1","addr":"172.21.15.204:6823","nonce":12360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":12360},{"type":"v1","addr":"172.21.15.204:6821","nonce":12360}]},"public_addr":"172.21.15.204:6817/12360","cluster_addr":"172.21.15.204:6819/12360","heartbeat_back_addr":"172.21.15.204:6823/12360","heartbeat_front_addr":"172.21.15.204:6821/12360","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-04-10T07:04:45.977943+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-04-10T07:04:46.313577+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-04-10T07:04:46.324207+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-04-10T07:04:55.035 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:55.039 INFO:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 --log-early tell osd.0 flush_pg_stats 2020-04-10T07:04:55.239 INFO:teuthology.orchestra.run.smithi204.stdout:38654705667 2020-04-10T07:04:55.239 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:55.244 INFO:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 --log-early tell osd.1 flush_pg_stats 2020-04-10T07:04:55.446 INFO:teuthology.orchestra.run.smithi204.stdout:38654705668 2020-04-10T07:04:55.446 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:55.451 INFO:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 --log-early tell osd.2 flush_pg_stats 2020-04-10T07:04:55.660 INFO:teuthology.orchestra.run.smithi204.stdout:38654705667 2020-04-10T07:04:55.661 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:55.666 INFO:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 --log-early osd last-stat-seq osd.0 2020-04-10T07:04:55.975 INFO:teuthology.orchestra.run.smithi204.stdout:38654705666 2020-04-10T07:04:55.996 INFO:tasks.ceph.ceph_manager.cluster2:need seq 38654705667 got 38654705666 for osd.0 2020-04-10T07:04:56.997 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:57.003 INFO:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 --log-early osd last-stat-seq osd.0 2020-04-10T07:04:57.310 INFO:teuthology.orchestra.run.smithi204.stdout:38654705667 2020-04-10T07:04:57.323 INFO:tasks.ceph.ceph_manager.cluster2:need seq 38654705667 got 38654705667 for osd.0 2020-04-10T07:04:57.323 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:57.329 INFO:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 --log-early osd last-stat-seq osd.1 2020-04-10T07:04:57.625 INFO:teuthology.orchestra.run.smithi204.stdout:38654705668 2020-04-10T07:04:57.638 INFO:tasks.ceph.ceph_manager.cluster2:need seq 38654705668 got 38654705668 for osd.1 2020-04-10T07:04:57.638 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:57.642 INFO:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 --log-early osd last-stat-seq osd.2 2020-04-10T07:04:57.942 INFO:teuthology.orchestra.run.smithi204.stdout:38654705667 2020-04-10T07:04:57.962 INFO:tasks.ceph.ceph_manager.cluster2:need seq 38654705667 got 38654705667 for osd.2 2020-04-10T07:04:57.962 INFO:tasks.ceph.ceph_manager.cluster2:waiting for clean 2020-04-10T07:04:57.962 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:57.967 INFO:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 --log-early pg dump --format=json 2020-04-10T07:04:58.266 INFO:teuthology.orchestra.run.smithi204.stdout: 2020-04-10T07:04:58.268 INFO:teuthology.orchestra.run.smithi204.stderr:dumped all 2020-04-10T07:04:58.280 INFO:teuthology.orchestra.run.smithi204.stdout:{"pg_ready":true,"pg_map":{"version":17,"stamp":"2020-04-10T07:04:57.784246+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":3146112,"kb_used_data":360,"kb_used_omap":0,"kb_used_meta":3145728,"kb_avail":279969408,"statfs":{"total":289910292480,"available":286688673792,"internally_reserved":3221225472,"allocated":368640,"data_stored":105681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":3221225472},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.770310"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":"9","reported_epoch":"13","state":"active+clean","last_fresh":"2020-04-10T07:04:53.027195+0000","last_change":"2020-04-10T07:04:53.024770+0000","last_active":"2020-04-10T07:04:53.027195+0000","last_peered":"2020-04-10T07:04:53.027195+0000","last_clean":"2020-04-10T07:04:53.027195+0000","last_became_active":"2020-04-10T07:04:53.024433+0000","last_became_peered":"2020-04-10T07:04:53.024433+0000","last_unstale":"2020-04-10T07:04:53.027195+0000","last_undegraded":"2020-04-10T07:04:53.027195+0000","last_fullsized":"2020-04-10T07:04:53.027195+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-04-10T07:04:52.001568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-04-10T07:04:52.001568+0000","last_clean_scrub_stamp":"2020-04-10T07:04:52.001568+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"9","reported_epoch":"13","state":"active+clean","last_fresh":"2020-04-10T07:04:53.027287+0000","last_change":"2020-04-10T07:04:53.025686+0000","last_active":"2020-04-10T07:04:53.027287+0000","last_peered":"2020-04-10T07:04:53.027287+0000","last_clean":"2020-04-10T07:04:53.027287+0000","last_became_active":"2020-04-10T07:04:53.025278+0000","last_became_peered":"2020-04-10T07:04:53.025278+0000","last_unstale":"2020-04-10T07:04:53.027287+0000","last_undegraded":"2020-04-10T07:04:53.027287+0000","last_fullsized":"2020-04-10T07:04:53.027287+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-04-10T07:04:52.001568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-04-10T07:04:52.001568+0000","last_clean_scrub_stamp":"2020-04-10T07:04:52.001568+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"9","reported_epoch":"13","state":"active+clean","last_fresh":"2020-04-10T07:04:53.027304+0000","last_change":"2020-04-10T07:04:53.026357+0000","last_active":"2020-04-10T07:04:53.027304+0000","last_peered":"2020-04-10T07:04:53.027304+0000","last_clean":"2020-04-10T07:04:53.027304+0000","last_became_active":"2020-04-10T07:04:53.025799+0000","last_became_peered":"2020-04-10T07:04:53.025799+0000","last_unstale":"2020-04-10T07:04:53.027304+0000","last_undegraded":"2020-04-10T07:04:53.027304+0000","last_fullsized":"2020-04-10T07:04:53.027304+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-04-10T07:04:52.001568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-04-10T07:04:52.001568+0000","last_clean_scrub_stamp":"2020-04-10T07:04:52.001568+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"9","reported_epoch":"13","state":"active+clean","last_fresh":"2020-04-10T07:04:53.024088+0000","last_change":"2020-04-10T07:04:53.023344+0000","last_active":"2020-04-10T07:04:53.024088+0000","last_peered":"2020-04-10T07:04:53.024088+0000","last_clean":"2020-04-10T07:04:53.024088+0000","last_became_active":"2020-04-10T07:04:53.022937+0000","last_became_peered":"2020-04-10T07:04:53.022937+0000","last_unstale":"2020-04-10T07:04:53.024088+0000","last_undegraded":"2020-04-10T07:04:53.024088+0000","last_fullsized":"2020-04-10T07:04:53.024088+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-04-10T07:04:52.001568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-04-10T07:04:52.001568+0000","last_clean_scrub_stamp":"2020-04-10T07:04:52.001568+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"10'1","reported_seq":"13","reported_epoch":"13","state":"active+clean","last_fresh":"2020-04-10T07:04:53.015120+0000","last_change":"2020-04-10T07:04:48.802435+0000","last_active":"2020-04-10T07:04:53.015120+0000","last_peered":"2020-04-10T07:04:53.015120+0000","last_clean":"2020-04-10T07:04:53.015120+0000","last_became_active":"2020-04-10T07:04:48.801883+0000","last_became_peered":"2020-04-10T07:04:48.801883+0000","last_unstale":"2020-04-10T07:04:53.015120+0000","last_undegraded":"2020-04-10T07:04:53.015120+0000","last_fullsized":"2020-04-10T07:04:53.015120+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":6,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-04-10T07:04:44.776661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-04-10T07:04:44.776661+0000","last_clean_scrub_stamp":"2020-04-10T07:04:44.776661+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"9","reported_epoch":"13","state":"active+clean","last_fresh":"2020-04-10T07:04:53.023242+0000","last_change":"2020-04-10T07:04:53.021081+0000","last_active":"2020-04-10T07:04:53.023242+0000","last_peered":"2020-04-10T07:04:53.023242+0000","last_clean":"2020-04-10T07:04:53.023242+0000","last_became_active":"2020-04-10T07:04:53.020650+0000","last_became_peered":"2020-04-10T07:04:53.020650+0000","last_unstale":"2020-04-10T07:04:53.023242+0000","last_undegraded":"2020-04-10T07:04:53.023242+0000","last_fullsized":"2020-04-10T07:04:53.023242+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-04-10T07:04:52.001568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-04-10T07:04:52.001568+0000","last_clean_scrub_stamp":"2020-04-10T07:04:52.001568+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"9","reported_epoch":"13","state":"active+clean","last_fresh":"2020-04-10T07:04:53.024803+0000","last_change":"2020-04-10T07:04:53.024232+0000","last_active":"2020-04-10T07:04:53.024803+0000","last_peered":"2020-04-10T07:04:53.024803+0000","last_clean":"2020-04-10T07:04:53.024803+0000","last_became_active":"2020-04-10T07:04:53.023869+0000","last_became_peered":"2020-04-10T07:04:53.023869+0000","last_unstale":"2020-04-10T07:04:53.024803+0000","last_undegraded":"2020-04-10T07:04:53.024803+0000","last_fullsized":"2020-04-10T07:04:53.024803+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-04-10T07:04:52.001568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-04-10T07:04:52.001568+0000","last_clean_scrub_stamp":"2020-04-10T07:04:52.001568+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"9","reported_epoch":"13","state":"active+clean","last_fresh":"2020-04-10T07:04:53.028499+0000","last_change":"2020-04-10T07:04:53.028028+0000","last_active":"2020-04-10T07:04:53.028499+0000","last_peered":"2020-04-10T07:04:53.028499+0000","last_clean":"2020-04-10T07:04:53.028499+0000","last_became_active":"2020-04-10T07:04:53.027681+0000","last_became_peered":"2020-04-10T07:04:53.027681+0000","last_unstale":"2020-04-10T07:04:53.028499+0000","last_undegraded":"2020-04-10T07:04:53.028499+0000","last_fullsized":"2020-04-10T07:04:53.028499+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-04-10T07:04:52.001568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-04-10T07:04:52.001568+0000","last_clean_scrub_stamp":"2020-04-10T07:04:52.001568+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"9","reported_epoch":"13","state":"active+clean","last_fresh":"2020-04-10T07:04:53.027425+0000","last_change":"2020-04-10T07:04:53.025924+0000","last_active":"2020-04-10T07:04:53.027425+0000","last_peered":"2020-04-10T07:04:53.027425+0000","last_clean":"2020-04-10T07:04:53.027425+0000","last_became_active":"2020-04-10T07:04:53.025544+0000","last_became_peered":"2020-04-10T07:04:53.025544+0000","last_unstale":"2020-04-10T07:04:53.027425+0000","last_undegraded":"2020-04-10T07:04:53.027425+0000","last_fullsized":"2020-04-10T07:04:53.027425+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-04-10T07:04:52.001568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-04-10T07:04:52.001568+0000","last_clean_scrub_stamp":"2020-04-10T07:04:52.001568+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":9,"seq":38654705667,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048704,"kb_used_data":120,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323136,"statfs":{"total":96636764160,"available":95562891264,"internally_reserved":1073741824,"allocated":122880,"data_stored":35227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.254}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6419999999999999}]}]},{"osd":0,"up_from":9,"seq":38654705667,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048704,"kb_used_data":120,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323136,"statfs":{"total":96636764160,"available":95562891264,"internally_reserved":1073741824,"allocated":122880,"data_stored":35227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.411}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3580000000000001}]}]},{"osd":1,"up_from":9,"seq":38654705668,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048704,"kb_used_data":120,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323136,"statfs":{"total":96636764160,"available":95562891264,"internally_reserved":1073741824,"allocated":122880,"data_stored":35227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1539999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98999999999999999}]}]}]}} 2020-04-10T07:04:58.281 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:58.286 INFO:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 --log-early pg dump --format=json 2020-04-10T07:04:58.588 INFO:teuthology.orchestra.run.smithi204.stdout: 2020-04-10T07:04:58.590 INFO:teuthology.orchestra.run.smithi204.stderr:dumped all 2020-04-10T07:04:58.601 INFO:teuthology.orchestra.run.smithi204.stdout:{"pg_ready":true,"pg_map":{"version":17,"stamp":"2020-04-10T07:04:57.784246+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":3146112,"kb_used_data":360,"kb_used_omap":0,"kb_used_meta":3145728,"kb_avail":279969408,"statfs":{"total":289910292480,"available":286688673792,"internally_reserved":3221225472,"allocated":368640,"data_stored":105681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":3221225472},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.770310"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":"9","reported_epoch":"13","state":"active+clean","last_fresh":"2020-04-10T07:04:53.027195+0000","last_change":"2020-04-10T07:04:53.024770+0000","last_active":"2020-04-10T07:04:53.027195+0000","last_peered":"2020-04-10T07:04:53.027195+0000","last_clean":"2020-04-10T07:04:53.027195+0000","last_became_active":"2020-04-10T07:04:53.024433+0000","last_became_peered":"2020-04-10T07:04:53.024433+0000","last_unstale":"2020-04-10T07:04:53.027195+0000","last_undegraded":"2020-04-10T07:04:53.027195+0000","last_fullsized":"2020-04-10T07:04:53.027195+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-04-10T07:04:52.001568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-04-10T07:04:52.001568+0000","last_clean_scrub_stamp":"2020-04-10T07:04:52.001568+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"9","reported_epoch":"13","state":"active+clean","last_fresh":"2020-04-10T07:04:53.027287+0000","last_change":"2020-04-10T07:04:53.025686+0000","last_active":"2020-04-10T07:04:53.027287+0000","last_peered":"2020-04-10T07:04:53.027287+0000","last_clean":"2020-04-10T07:04:53.027287+0000","last_became_active":"2020-04-10T07:04:53.025278+0000","last_became_peered":"2020-04-10T07:04:53.025278+0000","last_unstale":"2020-04-10T07:04:53.027287+0000","last_undegraded":"2020-04-10T07:04:53.027287+0000","last_fullsized":"2020-04-10T07:04:53.027287+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-04-10T07:04:52.001568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-04-10T07:04:52.001568+0000","last_clean_scrub_stamp":"2020-04-10T07:04:52.001568+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"9","reported_epoch":"13","state":"active+clean","last_fresh":"2020-04-10T07:04:53.027304+0000","last_change":"2020-04-10T07:04:53.026357+0000","last_active":"2020-04-10T07:04:53.027304+0000","last_peered":"2020-04-10T07:04:53.027304+0000","last_clean":"2020-04-10T07:04:53.027304+0000","last_became_active":"2020-04-10T07:04:53.025799+0000","last_became_peered":"2020-04-10T07:04:53.025799+0000","last_unstale":"2020-04-10T07:04:53.027304+0000","last_undegraded":"2020-04-10T07:04:53.027304+0000","last_fullsized":"2020-04-10T07:04:53.027304+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-04-10T07:04:52.001568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-04-10T07:04:52.001568+0000","last_clean_scrub_stamp":"2020-04-10T07:04:52.001568+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"9","reported_epoch":"13","state":"active+clean","last_fresh":"2020-04-10T07:04:53.024088+0000","last_change":"2020-04-10T07:04:53.023344+0000","last_active":"2020-04-10T07:04:53.024088+0000","last_peered":"2020-04-10T07:04:53.024088+0000","last_clean":"2020-04-10T07:04:53.024088+0000","last_became_active":"2020-04-10T07:04:53.022937+0000","last_became_peered":"2020-04-10T07:04:53.022937+0000","last_unstale":"2020-04-10T07:04:53.024088+0000","last_undegraded":"2020-04-10T07:04:53.024088+0000","last_fullsized":"2020-04-10T07:04:53.024088+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-04-10T07:04:52.001568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-04-10T07:04:52.001568+0000","last_clean_scrub_stamp":"2020-04-10T07:04:52.001568+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"10'1","reported_seq":"13","reported_epoch":"13","state":"active+clean","last_fresh":"2020-04-10T07:04:53.015120+0000","last_change":"2020-04-10T07:04:48.802435+0000","last_active":"2020-04-10T07:04:53.015120+0000","last_peered":"2020-04-10T07:04:53.015120+0000","last_clean":"2020-04-10T07:04:53.015120+0000","last_became_active":"2020-04-10T07:04:48.801883+0000","last_became_peered":"2020-04-10T07:04:48.801883+0000","last_unstale":"2020-04-10T07:04:53.015120+0000","last_undegraded":"2020-04-10T07:04:53.015120+0000","last_fullsized":"2020-04-10T07:04:53.015120+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":6,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-04-10T07:04:44.776661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-04-10T07:04:44.776661+0000","last_clean_scrub_stamp":"2020-04-10T07:04:44.776661+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"9","reported_epoch":"13","state":"active+clean","last_fresh":"2020-04-10T07:04:53.023242+0000","last_change":"2020-04-10T07:04:53.021081+0000","last_active":"2020-04-10T07:04:53.023242+0000","last_peered":"2020-04-10T07:04:53.023242+0000","last_clean":"2020-04-10T07:04:53.023242+0000","last_became_active":"2020-04-10T07:04:53.020650+0000","last_became_peered":"2020-04-10T07:04:53.020650+0000","last_unstale":"2020-04-10T07:04:53.023242+0000","last_undegraded":"2020-04-10T07:04:53.023242+0000","last_fullsized":"2020-04-10T07:04:53.023242+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-04-10T07:04:52.001568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-04-10T07:04:52.001568+0000","last_clean_scrub_stamp":"2020-04-10T07:04:52.001568+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"9","reported_epoch":"13","state":"active+clean","last_fresh":"2020-04-10T07:04:53.024803+0000","last_change":"2020-04-10T07:04:53.024232+0000","last_active":"2020-04-10T07:04:53.024803+0000","last_peered":"2020-04-10T07:04:53.024803+0000","last_clean":"2020-04-10T07:04:53.024803+0000","last_became_active":"2020-04-10T07:04:53.023869+0000","last_became_peered":"2020-04-10T07:04:53.023869+0000","last_unstale":"2020-04-10T07:04:53.024803+0000","last_undegraded":"2020-04-10T07:04:53.024803+0000","last_fullsized":"2020-04-10T07:04:53.024803+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-04-10T07:04:52.001568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-04-10T07:04:52.001568+0000","last_clean_scrub_stamp":"2020-04-10T07:04:52.001568+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"9","reported_epoch":"13","state":"active+clean","last_fresh":"2020-04-10T07:04:53.028499+0000","last_change":"2020-04-10T07:04:53.028028+0000","last_active":"2020-04-10T07:04:53.028499+0000","last_peered":"2020-04-10T07:04:53.028499+0000","last_clean":"2020-04-10T07:04:53.028499+0000","last_became_active":"2020-04-10T07:04:53.027681+0000","last_became_peered":"2020-04-10T07:04:53.027681+0000","last_unstale":"2020-04-10T07:04:53.028499+0000","last_undegraded":"2020-04-10T07:04:53.028499+0000","last_fullsized":"2020-04-10T07:04:53.028499+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-04-10T07:04:52.001568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-04-10T07:04:52.001568+0000","last_clean_scrub_stamp":"2020-04-10T07:04:52.001568+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"9","reported_epoch":"13","state":"active+clean","last_fresh":"2020-04-10T07:04:53.027425+0000","last_change":"2020-04-10T07:04:53.025924+0000","last_active":"2020-04-10T07:04:53.027425+0000","last_peered":"2020-04-10T07:04:53.027425+0000","last_clean":"2020-04-10T07:04:53.027425+0000","last_became_active":"2020-04-10T07:04:53.025544+0000","last_became_peered":"2020-04-10T07:04:53.025544+0000","last_unstale":"2020-04-10T07:04:53.027425+0000","last_undegraded":"2020-04-10T07:04:53.027425+0000","last_fullsized":"2020-04-10T07:04:53.027425+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-04-10T07:04:52.001568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-04-10T07:04:52.001568+0000","last_clean_scrub_stamp":"2020-04-10T07:04:52.001568+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":9,"seq":38654705667,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048704,"kb_used_data":120,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323136,"statfs":{"total":96636764160,"available":95562891264,"internally_reserved":1073741824,"allocated":122880,"data_stored":35227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.254}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6419999999999999}]}]},{"osd":0,"up_from":9,"seq":38654705667,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048704,"kb_used_data":120,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323136,"statfs":{"total":96636764160,"available":95562891264,"internally_reserved":1073741824,"allocated":122880,"data_stored":35227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.411}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3580000000000001}]}]},{"osd":1,"up_from":9,"seq":38654705668,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048704,"kb_used_data":120,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323136,"statfs":{"total":96636764160,"available":95562891264,"internally_reserved":1073741824,"allocated":122880,"data_stored":35227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1539999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98999999999999999}]}]}]}} 2020-04-10T07:04:58.602 INFO:tasks.ceph.ceph_manager.cluster2:clean! 2020-04-10T07:04:58.603 INFO:tasks.ceph:Waiting until ceph cluster cluster2 is healthy... 2020-04-10T07:04:58.603 INFO:tasks.ceph.ceph_manager.cluster2:wait_until_healthy 2020-04-10T07:04:58.603 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:58.608 INFO:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 --log-early health --format=json 2020-04-10T07:04:58.957 INFO:teuthology.orchestra.run.smithi204.stdout: 2020-04-10T07:04:58.970 INFO:teuthology.orchestra.run.smithi204.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2020-04-10T07:04:58.970 INFO:tasks.ceph.ceph_manager.cluster2:wait_until_healthy done 2020-04-10T07:04:58.971 INFO:teuthology.run_tasks:Running task exec... 2020-04-10T07:04:58.983 INFO:teuthology.task.exec:Executing custom commands... 2020-04-10T07:04:58.983 INFO:teuthology.task.exec:Running commands on role cluster1.client.mirror.0 host ubuntu@smithi204.front.sepia.ceph.com 2020-04-10T07:04:58.984 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:58.989 INFO:teuthology.orchestra.run.smithi204:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster1 auth caps client.mirror.4 mon '"'"'profile rbd-mirror'"'"' osd '"'"'profile rbd'"'"'' 2020-04-10T07:04:59.353 INFO:teuthology.orchestra.run.smithi204.stderr:updated caps for client.mirror.4 2020-04-10T07:04:59.365 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:59.370 INFO:teuthology.orchestra.run.smithi204:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster1 auth caps client.mirror.5 mon '"'"'profile rbd-mirror'"'"' osd '"'"'profile rbd'"'"'' 2020-04-10T07:04:59.742 INFO:teuthology.orchestra.run.smithi204.stderr:updated caps for client.mirror.5 2020-04-10T07:04:59.756 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:04:59.761 INFO:teuthology.orchestra.run.smithi204:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster1 auth caps client.mirror.6 mon '"'"'profile rbd-mirror'"'"' osd '"'"'profile rbd'"'"'' 2020-04-10T07:05:00.120 INFO:teuthology.orchestra.run.smithi204.stderr:updated caps for client.mirror.6 2020-04-10T07:05:00.132 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:05:00.137 INFO:teuthology.orchestra.run.smithi204:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster2 auth caps client.mirror.4 mon '"'"'profile rbd-mirror'"'"' osd '"'"'profile rbd'"'"'' 2020-04-10T07:05:00.512 INFO:teuthology.orchestra.run.smithi204.stderr:updated caps for client.mirror.4 2020-04-10T07:05:00.523 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:05:00.549 INFO:teuthology.orchestra.run.smithi204:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster2 auth caps client.mirror.5 mon '"'"'profile rbd-mirror'"'"' osd '"'"'profile rbd'"'"'' 2020-04-10T07:05:00.931 INFO:teuthology.orchestra.run.smithi204.stderr:updated caps for client.mirror.5 2020-04-10T07:05:00.943 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:05:00.950 INFO:teuthology.orchestra.run.smithi204:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster2 auth caps client.mirror.6 mon '"'"'profile rbd-mirror'"'"' osd '"'"'profile rbd'"'"'' 2020-04-10T07:05:01.328 INFO:teuthology.orchestra.run.smithi204.stderr:updated caps for client.mirror.6 2020-04-10T07:05:01.340 INFO:teuthology.run_tasks:Running task exec... 2020-04-10T07:05:01.359 INFO:teuthology.task.exec:Executing custom commands... 2020-04-10T07:05:01.359 INFO:teuthology.task.exec:Running commands on role cluster1.client.mirror.0 host ubuntu@smithi204.front.sepia.ceph.com 2020-04-10T07:05:01.359 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:05:01.375 INFO:teuthology.orchestra.run.smithi204:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster1 auth caps client.mirror mon '"'"'profile rbd-mirror-peer'"'"' osd '"'"'profile rbd'"'"'' 2020-04-10T07:05:01.738 INFO:teuthology.orchestra.run.smithi204.stderr:updated caps for client.mirror 2020-04-10T07:05:01.749 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:05:01.757 INFO:teuthology.orchestra.run.smithi204:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster1 auth caps client.mirror.0 mon '"'"'profile rbd-mirror'"'"' osd '"'"'profile rbd'"'"'' 2020-04-10T07:05:02.132 INFO:teuthology.orchestra.run.smithi204.stderr:updated caps for client.mirror.0 2020-04-10T07:05:02.143 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:05:02.147 INFO:teuthology.orchestra.run.smithi204:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster1 auth caps client.mirror.1 mon '"'"'profile rbd-mirror'"'"' osd '"'"'profile rbd'"'"'' 2020-04-10T07:05:02.511 INFO:teuthology.orchestra.run.smithi204.stderr:updated caps for client.mirror.1 2020-04-10T07:05:02.523 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:05:02.527 INFO:teuthology.orchestra.run.smithi204:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster1 auth caps client.mirror.2 mon '"'"'profile rbd-mirror'"'"' osd '"'"'profile rbd'"'"'' 2020-04-10T07:05:02.895 INFO:teuthology.orchestra.run.smithi204.stderr:updated caps for client.mirror.2 2020-04-10T07:05:02.907 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:05:02.912 INFO:teuthology.orchestra.run.smithi204:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster1 auth caps client.mirror.3 mon '"'"'profile rbd-mirror'"'"' osd '"'"'profile rbd'"'"'' 2020-04-10T07:05:03.281 INFO:teuthology.orchestra.run.smithi204.stderr:updated caps for client.mirror.3 2020-04-10T07:05:03.292 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:05:03.298 INFO:teuthology.orchestra.run.smithi204:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster2 auth caps client.mirror mon '"'"'profile rbd-mirror-peer'"'"' osd '"'"'profile rbd'"'"'' 2020-04-10T07:05:03.670 INFO:teuthology.orchestra.run.smithi204.stderr:updated caps for client.mirror 2020-04-10T07:05:03.681 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:05:03.688 INFO:teuthology.orchestra.run.smithi204:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster2 auth caps client.mirror.0 mon '"'"'profile rbd-mirror'"'"' osd '"'"'profile rbd'"'"'' 2020-04-10T07:05:04.047 INFO:teuthology.orchestra.run.smithi204.stderr:updated caps for client.mirror.0 2020-04-10T07:05:04.059 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:05:04.062 INFO:teuthology.orchestra.run.smithi204:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster2 auth caps client.mirror.1 mon '"'"'profile rbd-mirror'"'"' osd '"'"'profile rbd'"'"'' 2020-04-10T07:05:04.445 INFO:teuthology.orchestra.run.smithi204.stderr:updated caps for client.mirror.1 2020-04-10T07:05:04.462 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:05:04.465 INFO:teuthology.orchestra.run.smithi204:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster2 auth caps client.mirror.2 mon '"'"'profile rbd-mirror'"'"' osd '"'"'profile rbd'"'"'' 2020-04-10T07:05:04.850 INFO:teuthology.orchestra.run.smithi204.stderr:updated caps for client.mirror.2 2020-04-10T07:05:04.864 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:05:04.869 INFO:teuthology.orchestra.run.smithi204:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster2 auth caps client.mirror.3 mon '"'"'profile rbd-mirror'"'"' osd '"'"'profile rbd'"'"'' 2020-04-10T07:05:05.236 INFO:teuthology.orchestra.run.smithi204.stderr:updated caps for client.mirror.3 2020-04-10T07:05:05.249 INFO:teuthology.run_tasks:Running task workunit... 2020-04-10T07:05:05.270 INFO:tasks.workunit:Pulling workunits from ref 39357d2a782a02d1ea09c76cbe7bc6f222a1127a 2020-04-10T07:05:05.270 INFO:tasks.workunit:Making a separate scratch dir for every client... 2020-04-10T07:05:05.270 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:05:05.276 INFO:teuthology.orchestra.run.smithi204:> stat -- /home/ubuntu/cephtest/mnt.cluster1.mirror 2020-04-10T07:05:05.323 INFO:teuthology.orchestra.run.smithi204.stderr:stat: cannot stat '/home/ubuntu/cephtest/mnt.cluster1.mirror': No such file or directory 2020-04-10T07:05:05.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-04-10T07:05:05.324 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:05:05.370 INFO:teuthology.orchestra.run.smithi204:> mkdir -- /home/ubuntu/cephtest/mnt.cluster1.mirror 2020-04-10T07:05:05.421 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.cluster1.mirror 2020-04-10T07:05:05.421 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:05:05.466 INFO:teuthology.orchestra.run.smithi204:> cd -- /home/ubuntu/cephtest/mnt.cluster1.mirror && mkdir -- client.mirror 2020-04-10T07:05:05.516 INFO:tasks.workunit:timeout=3h 2020-04-10T07:05:05.516 INFO:tasks.workunit:cleanup=True 2020-04-10T07:05:05.517 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:05:05.562 INFO:teuthology.orchestra.run.smithi204:> rm -rf /home/ubuntu/cephtest/clone.cluster1.client.mirror && git clone git://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.cluster1.client.mirror && cd /home/ubuntu/cephtest/clone.cluster1.client.mirror && git checkout 39357d2a782a02d1ea09c76cbe7bc6f222a1127a 2020-04-10T07:05:05.667 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:Cloning into '/home/ubuntu/cephtest/clone.cluster1.client.mirror'... 2020-04-10T07:06:07.228 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:Note: checking out '39357d2a782a02d1ea09c76cbe7bc6f222a1127a'. 2020-04-10T07:06:07.229 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: 2020-04-10T07:06:07.229 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2020-04-10T07:06:07.229 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:changes and commit them, and you can discard any commits you make in this 2020-04-10T07:06:07.229 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:state without impacting any branches by performing another checkout. 2020-04-10T07:06:07.229 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: 2020-04-10T07:06:07.229 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:If you want to create a new branch to retain commits you create, you may 2020-04-10T07:06:07.230 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:do so (now or later) by using -b with the checkout command again. Example: 2020-04-10T07:06:07.230 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: 2020-04-10T07:06:07.230 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: git checkout -b 2020-04-10T07:06:07.230 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: 2020-04-10T07:06:07.230 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:HEAD is now at 39357d2a78 Merge pull request #34482 from ceph/octopus-fixes 2020-04-10T07:06:07.232 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:06:07.236 INFO:teuthology.orchestra.run.smithi204:> cd -- /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.cluster1.client.mirror 2020-04-10T07:06:07.286 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2020-04-10T07:06:07.289 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/direct_io' 2020-04-10T07:06:07.289 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2020-04-10T07:06:08.017 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2020-04-10T07:06:08.124 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2020-04-10T07:06:08.179 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/direct_io' 2020-04-10T07:06:08.180 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/fs' 2020-04-10T07:06:08.181 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2020-04-10T07:06:08.239 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/fs' 2020-04-10T07:06:08.292 DEBUG:teuthology.orchestra.remote:smithi204:/home/ubuntu/cephtest/workunits.list.cluster1.client.mirror is 4KB 2020-04-10T07:06:08.303 INFO:tasks.workunit:Running workunits matching rbd/rbd_mirror_snapshot.sh on cluster1.client.mirror... 2020-04-10T07:06:08.303 INFO:tasks.workunit:Running workunit rbd/rbd_mirror_snapshot.sh... 2020-04-10T07:06:08.303 INFO:teuthology.orchestra.run.smithi204:> true 2020-04-10T07:06:08.308 INFO:teuthology.orchestra.run.smithi204:workunit test rbd/rbd_mirror_snapshot.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd -- /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=39357d2a782a02d1ea09c76cbe7bc6f222a1127a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1" CEPH_ID="mirror" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_ARGS='' RBD_IMAGE_FEATURES=layering,exclusive-lock,journaling RBD_MIRROR_INSTANCES=4 RBD_MIRROR_CONFIG_KEY=1 RBD_MIRROR_USE_EXISTING_CLUSTER=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_snapshot.sh 2020-04-10T07:06:08.364 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ MIRROR_POOL_MODE=image 2020-04-10T07:06:08.365 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ MIRROR_IMAGE_MODE=snapshot 2020-04-10T07:06:08.365 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ dirname /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_snapshot.sh 2020-04-10T07:06:08.367 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ . /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_helpers.sh 2020-04-10T07:06:08.368 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ type xmlstarlet 2020-04-10T07:06:08.368 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ XMLSTARLET=xmlstarlet 2020-04-10T07:06:08.368 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ RBD_MIRROR_INSTANCES=4 2020-04-10T07:06:08.368 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CLUSTER1=cluster1 2020-04-10T07:06:08.369 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CLUSTER2=cluster2 2020-04-10T07:06:08.369 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ PEER_CLUSTER_SUFFIX= 2020-04-10T07:06:08.369 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ POOL=mirror 2020-04-10T07:06:08.369 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ PARENT_POOL=mirror_parent 2020-04-10T07:06:08.370 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ NS1=ns1 2020-04-10T07:06:08.370 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ NS2=ns2 2020-04-10T07:06:08.370 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ TEMPDIR= 2020-04-10T07:06:08.370 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ID=mirror 2020-04-10T07:06:08.370 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ RBD_IMAGE_FEATURES=layering,exclusive-lock,journaling 2020-04-10T07:06:08.371 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ MIRROR_USER_ID_PREFIX=mirror. 2020-04-10T07:06:08.371 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ MIRROR_POOL_MODE=image 2020-04-10T07:06:08.371 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ MIRROR_IMAGE_MODE=snapshot 2020-04-10T07:06:08.371 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ export CEPH_ARGS=--id mirror 2020-04-10T07:06:08.372 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ LAST_MIRROR_INSTANCE=3 2020-04-10T07:06:08.372 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ dirname /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_snapshot.sh 2020-04-10T07:06:08.372 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ readlink -f /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/../../../src 2020-04-10T07:06:08.376 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror/src 2020-04-10T07:06:08.376 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_BIN=. 2020-04-10T07:06:08.377 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_SRC=. 2020-04-10T07:06:08.377 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -e CMakeCache.txt ] 2020-04-10T07:06:08.377 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 -gt 0 ] 2020-04-10T07:06:08.377 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ setup 2020-04-10T07:06:08.377 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local c 2020-04-10T07:06:08.378 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ trap cleanup $? INT TERM EXIT 2020-04-10T07:06:08.378 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ setup_tempdir 2020-04-10T07:06:08.378 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:06:08.378 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mktemp -d 2020-04-10T07:06:08.378 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ TEMPDIR=/tmp/tmp.8SwBCsYPSC 2020-04-10T07:06:08.379 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -z 1 ] 2020-04-10T07:06:08.379 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ setup_pools cluster1 cluster2 2020-04-10T07:06:08.379 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:08.379 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local remote_cluster=cluster2 2020-04-10T07:06:08.380 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local mon_map_file 2020-04-10T07:06:08.380 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local mon_addr 2020-04-10T07:06:08.380 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local admin_key_file 2020-04-10T07:06:08.380 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local uuid 2020-04-10T07:06:08.380 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= ceph --cluster cluster1 osd pool create mirror 64 64 2020-04-10T07:06:09.117 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:pool 'mirror' already exists 2020-04-10T07:06:09.126 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= ceph --cluster cluster1 osd pool create mirror_parent 64 64 2020-04-10T07:06:10.123 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:pool 'mirror_parent' already exists 2020-04-10T07:06:10.132 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster1 pool init mirror 2020-04-10T07:06:13.062 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster1 pool init mirror_parent 2020-04-10T07:06:16.076 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n 1 ] 2020-04-10T07:06:16.077 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ PEER_CLUSTER_SUFFIX=-DNE 2020-04-10T07:06:16.077 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror pool enable --site-name cluster1-DNE mirror image 2020-04-10T07:06:16.134 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 mirror pool enable mirror_parent image 2020-04-10T07:06:16.173 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 namespace create mirror/ns1 2020-04-10T07:06:16.221 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 namespace create mirror/ns2 2020-04-10T07:06:16.264 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 mirror pool enable mirror/ns1 image 2020-04-10T07:06:16.318 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 mirror pool enable mirror/ns2 image 2020-04-10T07:06:16.371 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -z ] 2020-04-10T07:06:16.371 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -z 1 ] 2020-04-10T07:06:16.371 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mon_map_file=/tmp/tmp.8SwBCsYPSC/cluster2.monmap 2020-04-10T07:06:16.371 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= ceph --cluster cluster2 mon getmap 2020-04-10T07:06:16.788 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:got monmap epoch 1 2020-04-10T07:06:16.797 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ monmaptool --print /tmp/tmp.8SwBCsYPSC/cluster2.monmap 2020-04-10T07:06:16.798 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep -E mon\. 2020-04-10T07:06:16.798 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ head -n 1 2020-04-10T07:06:16.798 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -E s/^[0-9]+: ([^ ]+).+$/\1/ 2020-04-10T07:06:16.798 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -E s/\/[0-9]+//g 2020-04-10T07:06:16.817 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mon_addr=[v2:172.21.15.204:3300,v1:172.21.15.204:6789] 2020-04-10T07:06:16.817 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_key_file=/tmp/tmp.8SwBCsYPSC/cluster2.client.mirror.key 2020-04-10T07:06:16.817 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= ceph --cluster cluster2 auth get-key client.mirror 2020-04-10T07:06:17.222 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror pool peer add mirror client.mirror@cluster2-DNE --remote-mon-host [v2:172.21.15.204:3300,v1:172.21.15.204:6789] --remote-key-file /tmp/tmp.8SwBCsYPSC/cluster2.client.mirror.key 2020-04-10T07:06:17.272 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:2571f49c-d767-4112-ae12-d8b71263d58b 2020-04-10T07:06:17.275 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 mirror pool peer add mirror_parent client.mirror@cluster2-DNE 2020-04-10T07:06:17.317 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ uuid=467a3104-15d9-4d31-90d5-8005aa7c9eeb 2020-04-10T07:06:17.317 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror pool peer set mirror_parent 467a3104-15d9-4d31-90d5-8005aa7c9eeb mon-host [v2:172.21.15.204:3300,v1:172.21.15.204:6789] 2020-04-10T07:06:17.363 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror pool peer set mirror_parent 467a3104-15d9-4d31-90d5-8005aa7c9eeb key-file /tmp/tmp.8SwBCsYPSC/cluster2.client.mirror.key 2020-04-10T07:06:17.412 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ setup_pools cluster2 cluster1 2020-04-10T07:06:17.412 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:06:17.413 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local remote_cluster=cluster1 2020-04-10T07:06:17.413 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local mon_map_file 2020-04-10T07:06:17.413 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local mon_addr 2020-04-10T07:06:17.413 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local admin_key_file 2020-04-10T07:06:17.413 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local uuid 2020-04-10T07:06:17.413 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= ceph --cluster cluster2 osd pool create mirror 64 64 2020-04-10T07:06:18.206 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:pool 'mirror' already exists 2020-04-10T07:06:18.217 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= ceph --cluster cluster2 osd pool create mirror_parent 64 64 2020-04-10T07:06:19.242 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:pool 'mirror_parent' already exists 2020-04-10T07:06:19.250 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster2 pool init mirror 2020-04-10T07:06:22.156 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster2 pool init mirror_parent 2020-04-10T07:06:25.172 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n 1 ] 2020-04-10T07:06:25.172 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ PEER_CLUSTER_SUFFIX=-DNE 2020-04-10T07:06:25.172 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster2 mirror pool enable --site-name cluster2-DNE mirror image 2020-04-10T07:06:25.220 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 mirror pool enable mirror_parent image 2020-04-10T07:06:25.254 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 namespace create mirror/ns1 2020-04-10T07:06:25.290 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 namespace create mirror/ns2 2020-04-10T07:06:25.326 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 mirror pool enable mirror/ns1 image 2020-04-10T07:06:25.362 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 mirror pool enable mirror/ns2 image 2020-04-10T07:06:25.403 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -z ] 2020-04-10T07:06:25.403 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -z 1 ] 2020-04-10T07:06:25.403 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mon_map_file=/tmp/tmp.8SwBCsYPSC/cluster1.monmap 2020-04-10T07:06:25.403 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= ceph --cluster cluster1 mon getmap 2020-04-10T07:06:25.822 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:got monmap epoch 1 2020-04-10T07:06:25.832 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ monmaptool --print /tmp/tmp.8SwBCsYPSC/cluster1.monmap 2020-04-10T07:06:25.832 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep -E mon\. 2020-04-10T07:06:25.832 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ head -n 1 2020-04-10T07:06:25.832 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -E s/^[0-9]+: ([^ ]+).+$/\1/ 2020-04-10T07:06:25.832 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -E s/\/[0-9]+//g 2020-04-10T07:06:25.855 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mon_addr=[v2:172.21.15.101:3300,v1:172.21.15.101:6789] 2020-04-10T07:06:25.855 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_key_file=/tmp/tmp.8SwBCsYPSC/cluster1.client.mirror.key 2020-04-10T07:06:25.855 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= ceph --cluster cluster1 auth get-key client.mirror 2020-04-10T07:06:26.276 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster2 mirror pool peer add mirror client.mirror@cluster1-DNE --remote-mon-host [v2:172.21.15.101:3300,v1:172.21.15.101:6789] --remote-key-file /tmp/tmp.8SwBCsYPSC/cluster1.client.mirror.key 2020-04-10T07:06:26.339 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:90b6cfa2-5481-4edf-b462-5494aab62f6f 2020-04-10T07:06:26.341 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 mirror pool peer add mirror_parent client.mirror@cluster1-DNE 2020-04-10T07:06:26.371 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ uuid=39a2eac0-9f81-4c3b-a329-98e40ae52daf 2020-04-10T07:06:26.372 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster2 mirror pool peer set mirror_parent 39a2eac0-9f81-4c3b-a329-98e40ae52daf mon-host [v2:172.21.15.101:3300,v1:172.21.15.101:6789] 2020-04-10T07:06:26.415 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster2 mirror pool peer set mirror_parent 39a2eac0-9f81-4c3b-a329-98e40ae52daf key-file /tmp/tmp.8SwBCsYPSC/cluster1.client.mirror.key 2020-04-10T07:06:26.464 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:06:26.465 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ testlog TEST: add image and test replay 2020-04-10T07:06:26.465 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ tee+ -a /tmp/tmp.8SwBCsYPSC/rbd-mirror.test.log 2020-04-10T07:06:26.465 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:date +%F %T 2020-04-10T07:06:26.466 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10 07:06:26 TEST: add image and test replay 2020-04-10T07:06:26.466 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:2020-04-10 07:06:26 TEST: add image and test replay 2020-04-10T07:06:26.466 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ start_mirrors cluster1 2020-04-10T07:06:26.466 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:26.467 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:06:26.479 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ start_mirror cluster1:0 2020-04-10T07:06:26.479 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:06:26.479 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:26.480 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:06:26.480 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:06:26.480 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:06:26.480 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:26.480 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:26.480 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:06:26.480 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:06:26.481 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:06:26.481 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:26.481 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:06:26.481 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:06:26.481 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 2020-04-10T07:06:26.482 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd-mirror --cluster cluster1 --id mirror.0 --rbd-mirror-delete-retry-interval=5 --rbd-mirror-image-state-check-interval=5 --rbd-mirror-journal-poll-age=1 --rbd-mirror-pool-replayers-refresh-interval=5 --debug-rbd=30 --debug-journaler=30 --debug-rbd_mirror=30 --daemonize=true 2020-04-10T07:06:26.508 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ start_mirror cluster1:1 2020-04-10T07:06:26.508 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:1 2020-04-10T07:06:26.508 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:26.508 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 cluster instance 2020-04-10T07:06:26.508 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:06:26.509 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:06:26.509 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:26.509 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:26.509 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:06:26.509 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:06:26.509 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:06:26.509 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:26.510 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:06:26.510 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:06:26.510 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 2020-04-10T07:06:26.510 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd-mirror --cluster cluster1 --id mirror.1 --rbd-mirror-delete-retry-interval=5 --rbd-mirror-image-state-check-interval=5 --rbd-mirror-journal-poll-age=1 --rbd-mirror-pool-replayers-refresh-interval=5 --debug-rbd=30 --debug-journaler=30 --debug-rbd_mirror=30 --daemonize=true 2020-04-10T07:06:26.533 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ start_mirror cluster1:2 2020-04-10T07:06:26.534 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:2 2020-04-10T07:06:26.534 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:26.534 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 cluster instance 2020-04-10T07:06:26.534 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:06:26.534 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:06:26.534 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:26.535 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:26.535 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:06:26.535 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:06:26.535 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:06:26.535 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:26.535 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:06:26.535 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:06:26.535 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 2020-04-10T07:06:26.536 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd-mirror --cluster cluster1 --id mirror.2 --rbd-mirror-delete-retry-interval=5 --rbd-mirror-image-state-check-interval=5 --rbd-mirror-journal-poll-age=1 --rbd-mirror-pool-replayers-refresh-interval=5 --debug-rbd=30 --debug-journaler=30 --debug-rbd_mirror=30 --daemonize=true 2020-04-10T07:06:26.563 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ start_mirror cluster1:3 2020-04-10T07:06:26.563 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:3 2020-04-10T07:06:26.563 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:26.564 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:3 cluster instance 2020-04-10T07:06:26.564 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:3 2020-04-10T07:06:26.564 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:06:26.564 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:26.564 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:26.564 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:06:26.564 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 3 = cluster1:3 ] 2020-04-10T07:06:26.565 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:06:26.565 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:26.565 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=3 2020-04-10T07:06:26.565 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:06:26.565 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 2020-04-10T07:06:26.565 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd-mirror --cluster cluster1 --id mirror.3 --rbd-mirror-delete-retry-interval=5 --rbd-mirror-image-state-check-interval=5 --rbd-mirror-journal-poll-age=1 --rbd-mirror-pool-replayers-refresh-interval=5 --debug-rbd=30 --debug-journaler=30 --debug-rbd_mirror=30 --daemonize=true 2020-04-10T07:06:26.595 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ image=test 2020-04-10T07:06:26.596 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ create_image_and_enable_mirror cluster2 mirror test 2020-04-10T07:06:26.596 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:06:26.596 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:06:26.596 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:26.596 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:06:26.596 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:06:26.597 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:06:26.597 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local mode=snapshot 2020-04-10T07:06:26.597 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:06:26.597 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ create_image cluster2 mirror test 2020-04-10T07:06:26.597 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:06:26.597 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:06:26.597 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:26.598 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:06:26.598 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:06:26.598 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:06:26.598 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local size=128 2020-04-10T07:06:26.598 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:06:26.598 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 create --size 128 --image-feature layering,exclusive-lock,journaling mirror/test 2020-04-10T07:06:26.649 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ enable_mirror cluster2 mirror test snapshot 2020-04-10T07:06:26.649 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:06:26.649 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:26.649 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:06:26.650 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local mode=snapshot 2020-04-10T07:06:26.650 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/test snapshot 2020-04-10T07:06:27.497 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Mirroring enabled 2020-04-10T07:06:27.507 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_image_meta cluster2 mirror test key1 value1 2020-04-10T07:06:27.507 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:06:27.507 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:27.507 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:06:27.507 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local key=key1 2020-04-10T07:06:27.508 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=value1 2020-04-10T07:06:27.508 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 image-meta set mirror/test key1 value1 2020-04-10T07:06:27.588 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_image_meta cluster2 mirror test key2 value2 2020-04-10T07:06:27.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:06:27.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:27.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:06:27.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local key=key2 2020-04-10T07:06:27.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=value2 2020-04-10T07:06:27.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 image-meta set mirror/test key2 value2 2020-04-10T07:06:27.670 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_started cluster1 mirror test 2020-04-10T07:06:27.671 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:27.671 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:27.671 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:06:27.671 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2020-04-10T07:06:27.671 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:27.671 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:27.671 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:06:27.672 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state=started 2020-04-10T07:06:27.672 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local s 2020-04-10T07:06:27.672 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:06:28.671 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_image_replay_state cluster1 mirror test started 2020-04-10T07:06:28.672 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:28.672 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:28.672 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:06:28.672 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local test_state=started 2020-04-10T07:06:28.672 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_result 2020-04-10T07:06:28.672 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_state=stopped 2020-04-10T07:06:28.674 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2020-04-10T07:06:28.675 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_instance=cluster1 2020-04-10T07:06:28.675 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:06:28.676 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ + local cluster=cluster1 2020-04-10T07:06:28.676 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ greplocal -i state 2020-04-10T07:06:28.676 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: instance=cluster1 2020-04-10T07:06:28.676 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local loop_instance 2020-04-10T07:06:28.676 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0 2020-04-10T07:06:28.676 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ cluster1 != cluster1 ] 2020-04-10T07:06:28.676 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:06:28.678 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2020-04-10T07:06:28.678 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:06:28.678 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:06:28.679 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:28.679 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:06:28.679 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:06:28.679 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:06:28.679 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:28.679 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:28.679 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:06:28.680 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:06:28.680 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:06:28.680 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:28.680 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:06:28.680 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:06:28.680 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster1 2020-04-10T07:06:28.682 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:06:28.682 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:28.682 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:28.682 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:06:28.682 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:06:28.682 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:06:28.683 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:28.683 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:28.683 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:06:28.683 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:06:28.683 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:06:28.683 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:06:28.683 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:06:28.684 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:06:28.684 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:06:28.699 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:06:28.699 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:06:28.699 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:06:28.699 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2020-04-10T07:06:28.794 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:Invalid command: unused arguments: ['mirror/test'] 2020-04-10T07:06:28.794 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status : get status for rbd mirror 2020-04-10T07:06:28.794 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:06:28.796 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2020-04-10T07:06:28.796 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:1 2020-04-10T07:06:28.797 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:06:28.797 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:28.797 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 cluster instance 2020-04-10T07:06:28.797 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:06:28.797 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:06:28.797 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:28.797 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:28.798 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:06:28.798 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:06:28.798 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:06:28.798 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:28.798 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:06:28.798 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:06:28.798 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:1 cluster1 2020-04-10T07:06:28.799 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:1 2020-04-10T07:06:28.799 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:28.799 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:28.799 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2020-04-10T07:06:28.799 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:06:28.799 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:06:28.799 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:28.800 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:28.800 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:06:28.800 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:06:28.800 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:06:28.800 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:06:28.800 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:06:28.800 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:06:28.801 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2020-04-10T07:06:28.817 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:06:28.817 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:06:28.817 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:06:28.817 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2020-04-10T07:06:28.909 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:Invalid command: unused arguments: ['mirror/test'] 2020-04-10T07:06:28.909 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status : get status for rbd mirror 2020-04-10T07:06:28.909 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:06:28.911 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:2 rbd mirror status mirror/test 2020-04-10T07:06:28.911 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:2 2020-04-10T07:06:28.911 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:06:28.912 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:28.912 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 cluster instance 2020-04-10T07:06:28.912 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:06:28.912 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:06:28.912 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:28.912 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:28.913 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:06:28.913 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:06:28.913 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:06:28.913 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:28.913 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:06:28.913 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:06:28.913 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:2 cluster1 2020-04-10T07:06:28.913 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:2 2020-04-10T07:06:28.914 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:28.914 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:28.914 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2020-04-10T07:06:28.914 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:06:28.914 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:06:28.914 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:28.914 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:28.915 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:06:28.915 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:06:28.915 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:06:28.915 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:06:28.915 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:06:28.915 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:06:28.915 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2020-04-10T07:06:28.932 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:06:28.932 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:06:28.932 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:06:28.932 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2020-04-10T07:06:29.019 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:Invalid command: unused arguments: ['mirror/test'] 2020-04-10T07:06:29.019 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status : get status for rbd mirror 2020-04-10T07:06:29.019 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:06:29.022 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:3 rbd mirror status mirror/test 2020-04-10T07:06:29.022 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:3 2020-04-10T07:06:29.022 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:06:29.022 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:29.022 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:3 cluster instance 2020-04-10T07:06:29.022 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:3 2020-04-10T07:06:29.023 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:06:29.023 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:29.023 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:29.023 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:06:29.023 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 3 = cluster1:3 ] 2020-04-10T07:06:29.023 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:06:29.023 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:29.023 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=3 2020-04-10T07:06:29.024 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:06:29.024 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:3 cluster1 2020-04-10T07:06:29.024 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:3 2020-04-10T07:06:29.024 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:29.024 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:29.024 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:3 local_cluster instance 2020-04-10T07:06:29.024 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:3 2020-04-10T07:06:29.025 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:06:29.025 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:29.025 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:29.025 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:06:29.025 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 3 = cluster1:3 ] 2020-04-10T07:06:29.025 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:06:29.025 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:06:29.026 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=3 2020-04-10T07:06:29.026 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:06:29.026 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.3 admin socket 2020-04-10T07:06:29.035 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:06:29.036 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:06:29.036 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:06:29.036 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/test 2020-04-10T07:06:29.115 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:Invalid command: unused arguments: ['mirror/test'] 2020-04-10T07:06:29.116 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status : get status for rbd mirror 2020-04-10T07:06:29.116 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:06:29.116 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:06:30.118 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ cluster1 != cluster1 ] 2020-04-10T07:06:30.118 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:06:30.120 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2020-04-10T07:06:30.120 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:06:30.120 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:06:30.120 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:30.121 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:06:30.121 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:06:30.121 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:06:30.121 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:30.121 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:30.121 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:06:30.121 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:06:30.122 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:06:30.122 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:30.122 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:06:30.122 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:06:30.122 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster1 2020-04-10T07:06:30.122 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:06:30.122 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:30.122 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:30.123 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:06:30.123 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:06:30.123 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:06:30.123 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:30.123 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:30.123 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:06:30.123 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:06:30.124 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:06:30.124 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:06:30.124 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:06:30.124 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:06:30.124 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:06:30.141 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:06:30.142 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:06:30.142 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:06:30.142 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2020-04-10T07:06:30.236 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:06:30.237 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ status_result= "state": "Replaying" 2020-04-10T07:06:30.237 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo "state": "Replaying" 2020-04-10T07:06:30.237 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep -i Replaying 2020-04-10T07:06:30.242 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "state": "Replaying" 2020-04-10T07:06:30.242 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ current_state=started 2020-04-10T07:06:30.242 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test started = started 2020-04-10T07:06:30.242 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:06:30.242 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ write_image cluster2 mirror test 100 2020-04-10T07:06:30.242 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:06:30.243 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:30.243 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:06:30.243 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local count=100 2020-04-10T07:06:30.243 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local size= 2020-04-10T07:06:30.243 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 2020-04-10T07:06:30.243 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ size=4096 2020-04-10T07:06:30.244 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 bench mirror/test --io-type write --io-size 4096 --io-threads 1 --io-total 409600 --io-pattern rand 2020-04-10T07:06:30.314 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:bench type write io_size 4096 io_threads 1 bytes 409600 pattern random 2020-04-10T07:06:30.381 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: SEC OPS OPS/SEC BYTES/SEC 2020-04-10T07:06:30.382 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:elapsed: 0 ops: 100 ops/sec: 1470.59 bytes/sec: 5.7 MiB/s 2020-04-10T07:06:30.394 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror test 2020-04-10T07:06:30.394 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1 2020-04-10T07:06:30.394 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:06:30.394 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:30.394 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:06:30.394 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ snapshot = journal ] 2020-04-10T07:06:30.395 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ snapshot = snapshot ] 2020-04-10T07:06:30.395 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror test 2020-04-10T07:06:30.395 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1 2020-04-10T07:06:30.395 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:06:30.395 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:30.395 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:06:30.395 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster2-mirror-test.status 2020-04-10T07:06:30.396 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2-mirror-test.status 2020-04-10T07:06:30.396 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:06:30.396 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.status 2020-04-10T07:06:30.400 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror-test.status 2020-04-10T07:06:30.401 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror-test.status 2020-04-10T07:06:30.401 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:06:30.401 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.status 2020-04-10T07:06:30.401 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mirror_image_snapshot cluster2 mirror test 2020-04-10T07:06:30.401 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:06:30.401 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:30.401 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:06:30.402 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test 2020-04-10T07:06:30.901 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Snapshot ID: 5 2020-04-10T07:06:30.911 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_newest_mirror_snapshot cluster2 mirror test /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.status 2020-04-10T07:06:30.912 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:06:30.912 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:30.912 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:06:30.912 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local log=/tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.status 2020-04-10T07:06:30.912 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 snap list --all mirror/test --format xml 2020-04-10T07:06:30.912 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2020-04-10T07:06:31.015 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //snapshot/id 2020-04-10T07:06:31.021 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snapshot_id=5 2020-04-10T07:06:31.021 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ true 2020-04-10T07:06:31.021 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0.2 2020-04-10T07:06:31.223 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.status 2020-04-10T07:06:31.223 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:31.223 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:31.224 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:06:31.224 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.status 2020-04-10T07:06:31.224 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2020-04-10T07:06:31.224 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2020-04-10T07:06:31.293 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2020-04-10T07:06:31.297 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local primary_snapshot_id=5 2020-04-10T07:06:31.297 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test 5 = 5 2020-04-10T07:06:31.297 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:06:31.297 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2020-04-10T07:06:31.298 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:31.298 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:31.298 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:06:31.298 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:06:31.298 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:06:31.299 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:06:31.299 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:06:32.298 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying 2020-04-10T07:06:32.299 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:32.299 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:32.299 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:06:32.299 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:06:32.299 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:06:32.300 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:06:32.300 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror-test.mirror_status 2020-04-10T07:06:32.301 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror-test.mirror_status 2020-04-10T07:06:32.301 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:06:32.302 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:06:32.302 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test 2020-04-10T07:06:32.302 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ tee /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:06:32.370 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:test: 2020-04-10T07:06:32.370 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: global_id: 687e9f7d-ae46-4324-b8f7-7c16605e0fa6 2020-04-10T07:06:32.370 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+replaying 2020-04-10T07:06:32.371 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: replaying, {"replay_state": "idle", "remote_snapshot_timestamp": 1586502387} 2020-04-10T07:06:32.371 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: service: mirror.0 on smithi204 2020-04-10T07:06:32.371 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:06:29 2020-04-10T07:06:32.378 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ state: .*up+replaying /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:06:32.379 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+replaying 2020-04-10T07:06:32.380 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ description: .* /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:06:32.381 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: replaying, {"replay_state": "idle", "remote_snapshot_timestamp": 1586502387} 2020-04-10T07:06:32.382 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:06:32.382 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+replaying 2020-04-10T07:06:32.382 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^up+ 2020-04-10T07:06:32.385 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+replaying 2020-04-10T07:06:32.385 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: service: mirror.0 on smithi204 2020-04-10T07:06:32.385 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep service: *mirror..* on /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:06:32.390 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:06:32.392 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update=2020-04-10 07:06:29 2020-04-10T07:06:32.394 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying 2020-04-10 07:06:29 2020-04-10T07:06:32.394 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:32.394 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:32.394 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:06:32.394 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:06:32.394 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local prev_last_update=2020-04-10 07:06:29 2020-04-10T07:06:32.394 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror.mirror_status 2020-04-10T07:06:32.395 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror.mirror_status 2020-04-10T07:06:32.395 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:06:32.395 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror.mirror_status 2020-04-10T07:06:32.395 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2020-04-10T07:06:32.431 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: failed to query services: (13) Permission denied 2020-04-10T07:06:32.477 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update state 2020-04-10T07:06:32.478 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2020-04-10T07:06:32.483 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ last_update=2020-04-10 07:06:29 2020-04-10T07:06:32.484 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2020-04-10T07:06:32.488 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ state=up+replaying 2020-04-10T07:06:32.489 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+replaying 2020-04-10T07:06:32.489 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep up+replaying 2020-04-10T07:06:32.489 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+replaying 2020-04-10T07:06:32.491 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:06:32.491 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:06:32.491 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -z ] 2020-04-10T07:06:32.491 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_status_in_pool_dir cluster2 mirror test down+unknown 2020-04-10T07:06:32.491 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:06:32.492 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:32.492 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:06:32.492 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=down+unknown 2020-04-10T07:06:32.492 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:06:32.492 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:06:32.492 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:06:33.492 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_status_in_pool_dir cluster2 mirror test down+unknown 2020-04-10T07:06:33.492 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:06:33.492 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:33.492 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:06:33.493 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=down+unknown 2020-04-10T07:06:33.493 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:06:33.493 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:06:33.493 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster2-mirror-test.mirror_status 2020-04-10T07:06:33.494 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2-mirror-test.mirror_status 2020-04-10T07:06:33.494 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:06:33.494 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:06:33.495 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster2 mirror image status mirror/test 2020-04-10T07:06:33.495 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ tee /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:06:33.556 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:test: 2020-04-10T07:06:33.557 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: global_id: 687e9f7d-ae46-4324-b8f7-7c16605e0fa6 2020-04-10T07:06:33.557 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: down+unknown 2020-04-10T07:06:33.557 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: status not found 2020-04-10T07:06:33.557 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10T07:06:33.557 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: peer_sites: 2020-04-10T07:06:33.557 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: name: cluster1-DNE 2020-04-10T07:06:33.558 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+replaying 2020-04-10T07:06:33.558 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: replaying, {"replay_state": "idle", "remote_snapshot_timestamp": 1586502387} 2020-04-10T07:06:33.558 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:06:29 2020-04-10T07:06:33.558 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: snapshots: 2020-04-10T07:06:33.558 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: 5 .mirror.primary.687e9f7d-ae46-4324-b8f7-7c16605e0fa6.b120f20b-d90a-4c1d-a5bf-a95d0c415304 (peer_uuids:[90b6cfa2-5481-4edf-b462-5494aab62f6f]) 2020-04-10T07:06:33.564 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ state: .*down+unknown /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:06:33.566 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: down+unknown 2020-04-10T07:06:33.566 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ description: .* /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:06:33.567 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: status not found 2020-04-10T07:06:33.567 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:06:33.568 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo down+unknown 2020-04-10T07:06:33.568 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^up+ 2020-04-10T07:06:33.568 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep service: /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:06:33.570 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:06:33.572 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update= 2020-04-10T07:06:33.572 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_mirror_pool_status_verbose cluster2 mirror test down+unknown 2020-04-10T07:06:33.572 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:06:33.572 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:33.572 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:06:33.573 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=down+unknown 2020-04-10T07:06:33.573 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local prev_last_update= 2020-04-10T07:06:33.573 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster2-mirror.mirror_status 2020-04-10T07:06:33.573 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2-mirror.mirror_status+ 2020-04-10T07:06:33.573 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:sed -e s|[/ ]|_|g 2020-04-10T07:06:33.574 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster2-mirror.mirror_status 2020-04-10T07:06:33.574 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2020-04-10T07:06:33.616 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: failed to query services: (13) Permission denied 2020-04-10T07:06:33.652 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update state 2020-04-10T07:06:33.652 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2020-04-10T07:06:33.655 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ last_update= 2020-04-10T07:06:33.655 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2020-04-10T07:06:33.658 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ state=down+unknown 2020-04-10T07:06:33.658 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo down+unknown 2020-04-10T07:06:33.658 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep down+unknown 2020-04-10T07:06:33.660 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:down+unknown 2020-04-10T07:06:33.660 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:06:33.660 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:06:33.660 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ compare_images mirror test 2020-04-10T07:06:33.660 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:33.660 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:06:33.661 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local ret=0 2020-04-10T07:06:33.661 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster2-mirror-test.export 2020-04-10T07:06:33.661 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2-mirror-test.export 2020-04-10T07:06:33.661 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:06:33.662 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local rmt_export=/tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.export 2020-04-10T07:06:33.662 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror-test.export 2020-04-10T07:06:33.663 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror-test.export 2020-04-10T07:06:33.664 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:06:33.664 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local loc_export=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.export 2020-04-10T07:06:33.664 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rm -f /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.export /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.export 2020-04-10T07:06:33.664 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 export mirror/test /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.export 2020-04-10T07:06:33.838 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2020-04-10T07:06:33.844 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 export mirror/test /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.export 2020-04-10T07:06:34.010 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2020-04-10T07:06:34.014 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cmp /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.export /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.export 2020-04-10T07:06:34.088 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rm -f /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.export /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.export 2020-04-10T07:06:34.119 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:06:34.119 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ compare_image_meta cluster1 mirror test key1 value1 2020-04-10T07:06:34.119 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:34.119 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:34.120 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:06:34.120 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local key=key1 2020-04-10T07:06:34.120 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local value=value1 2020-04-10T07:06:34.120 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 image-meta get mirror/test key1 2020-04-10T07:06:34.187 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test value1 = value1 2020-04-10T07:06:34.187 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ compare_image_meta cluster1 mirror test key2 value2 2020-04-10T07:06:34.187 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:34.187 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:34.188 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:06:34.188 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local key=key2 2020-04-10T07:06:34.188 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local value=value2 2020-04-10T07:06:34.188 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 image-meta get mirror/test key2 2020-04-10T07:06:34.252 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test value2 = value2 2020-04-10T07:06:34.252 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ testlog TEST: stop mirror, add image, start mirror and test replay 2020-04-10T07:06:34.252 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ tee -a /tmp/tmp.8SwBCsYPSC/rbd-mirror.test.log 2020-04-10T07:06:34.253 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ date +%F %T 2020-04-10T07:06:34.258 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10 07:06:34 TEST: stop mirror, add image, start mirror and test replay 2020-04-10T07:06:34.258 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:2020-04-10 07:06:34 TEST: stop mirror, add image, start mirror and test replay 2020-04-10T07:06:34.259 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ stop_mirrors cluster1 2020-04-10T07:06:34.259 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:34.259 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local sig= 2020-04-10T07:06:34.259 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:06:34.259 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ stop_mirror cluster1:0 2020-04-10T07:06:34.260 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:06:34.260 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local sig= 2020-04-10T07:06:34.260 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 2020-04-10T07:06:34.260 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pid 2020-04-10T07:06:34.261 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_pid_file cluster1:0 2020-04-10T07:06:34.261 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:06:34.261 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:34.261 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:06:34.262 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:06:34.262 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:06:34.262 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:34.262 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:34.262 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:06:34.263 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:06:34.263 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:06:34.263 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:34.266 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:06:34.267 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:06:34.269 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 pid file 2020-04-10T07:06:34.276 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.pid 2020-04-10T07:06:34.277 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cat /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.pid 2020-04-10T07:06:34.277 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ pid=14588 2020-04-10T07:06:34.277 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n 14588 ] 2020-04-10T07:06:34.277 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ kill 14588 2020-04-10T07:06:34.278 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:06:35.278 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ps auxww 2020-04-10T07:06:35.278 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ awk -v pid=14588 $2 == pid {print; exit 1} 2020-04-10T07:06:35.296 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ break 2020-04-10T07:06:35.296 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ + awk -v pid=14588 $2 == pid {print; exit 1} 2020-04-10T07:06:35.296 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:ps auxww 2020-04-10T07:06:35.316 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster1 2020-04-10T07:06:35.316 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:06:35.316 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:35.316 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:35.317 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:06:35.317 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:06:35.317 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:06:35.317 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:35.317 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:35.317 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:06:35.317 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:06:35.318 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:06:35.318 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:06:35.318 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:06:35.318 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:06:35.318 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:06:35.336 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:06:35.337 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rm -f /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:06:35.338 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster2 2020-04-10T07:06:35.338 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:06:35.338 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:06:35.338 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:35.339 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:06:35.339 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:06:35.339 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:06:35.339 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:35.339 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:35.339 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:06:35.339 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:06:35.339 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:06:35.340 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:06:35.340 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:06:35.340 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:06:35.341 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:06:35.358 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:06:35.358 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rm -f /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:06:35.359 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_pid_file cluster1:0 2020-04-10T07:06:35.359 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:06:35.359 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:35.360 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:06:35.360 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:06:35.360 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:06:35.360 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:35.360 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:35.360 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:06:35.361 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:06:35.361 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:06:35.361 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:35.361 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:06:35.361 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:06:35.361 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 pid file 2020-04-10T07:06:35.381 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.pid 2020-04-10T07:06:35.381 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rm -f /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.pid 2020-04-10T07:06:35.384 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ stop_mirror cluster1:1 2020-04-10T07:06:35.384 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:1 2020-04-10T07:06:35.384 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local sig= 2020-04-10T07:06:35.384 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 2020-04-10T07:06:35.384 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pid 2020-04-10T07:06:35.384 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_pid_file cluster1:1 2020-04-10T07:06:35.384 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:1 2020-04-10T07:06:35.385 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:35.385 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 cluster instance 2020-04-10T07:06:35.385 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:06:35.385 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:06:35.385 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:35.385 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:35.385 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:06:35.386 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:06:35.386 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:06:35.386 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:35.386 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:06:35.386 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:06:35.386 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 pid file 2020-04-10T07:06:35.404 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.pid 2020-04-10T07:06:35.404 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cat /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.pid 2020-04-10T07:06:35.405 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ pid=14619 2020-04-10T07:06:35.405 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n 14619 ] 2020-04-10T07:06:35.406 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ kill 14619 2020-04-10T07:06:35.406 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:06:36.407 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ps auxww 2020-04-10T07:06:36.408 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ awk -v pid=14619 $2 == pid {print; exit 1} 2020-04-10T07:06:36.427 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ break 2020-04-10T07:06:36.428 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ps auxww 2020-04-10T07:06:36.428 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ awk -v pid=14619 $2 == pid {print; exit 1} 2020-04-10T07:06:36.448 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:1 cluster1 2020-04-10T07:06:36.448 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:1 2020-04-10T07:06:36.448 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:36.448 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:36.448 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2020-04-10T07:06:36.449 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:06:36.449 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:06:36.449 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:36.449 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:36.449 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:06:36.449 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:06:36.449 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:06:36.450 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:06:36.450 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:06:36.450 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:06:36.450 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2020-04-10T07:06:36.472 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:06:36.472 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rm -f /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:06:36.474 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:1 cluster2 2020-04-10T07:06:36.474 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:1 2020-04-10T07:06:36.474 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:06:36.474 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:36.475 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2020-04-10T07:06:36.475 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:06:36.475 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:06:36.475 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:36.475 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:36.475 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:06:36.475 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:06:36.475 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:06:36.476 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:06:36.476 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:06:36.476 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:06:36.476 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2020-04-10T07:06:36.497 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:06:36.497 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rm -f /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:06:36.499 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_pid_file cluster1:1 2020-04-10T07:06:36.499 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:1 2020-04-10T07:06:36.499 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:36.499 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 cluster instance 2020-04-10T07:06:36.499 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:06:36.499 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:06:36.500 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:36.500 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:36.500 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:06:36.500 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:06:36.500 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:06:36.500 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:36.500 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:06:36.501 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:06:36.501 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 pid file 2020-04-10T07:06:36.518 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.pid 2020-04-10T07:06:36.518 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rm -f /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.pid 2020-04-10T07:06:36.521 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ stop_mirror cluster1:2 2020-04-10T07:06:36.521 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:2 2020-04-10T07:06:36.521 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local sig= 2020-04-10T07:06:36.521 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 2020-04-10T07:06:36.521 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pid 2020-04-10T07:06:36.521 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_pid_file cluster1:2 2020-04-10T07:06:36.522 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:2 2020-04-10T07:06:36.522 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:36.528 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 cluster instance 2020-04-10T07:06:36.528 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:06:36.529 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:06:36.529 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:36.529 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:36.529 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:06:36.529 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:06:36.529 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:06:36.529 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:36.530 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:06:36.530 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:06:36.530 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 pid file 2020-04-10T07:06:36.542 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.pid 2020-04-10T07:06:36.542 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cat /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.pid 2020-04-10T07:06:36.543 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ pid=14650 2020-04-10T07:06:36.543 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n 14650 ] 2020-04-10T07:06:36.543 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ kill 14650 2020-04-10T07:06:36.543 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:06:37.545 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ps auxww 2020-04-10T07:06:37.545 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ awk -v pid=14650 $2 == pid {print; exit 1} 2020-04-10T07:06:37.564 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ break 2020-04-10T07:06:37.564 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ps auxww 2020-04-10T07:06:37.564 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ awk -v pid=14650 $2 == pid {print; exit 1} 2020-04-10T07:06:37.583 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:2 cluster1 2020-04-10T07:06:37.583 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:2 2020-04-10T07:06:37.583 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:37.584 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:37.584 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2020-04-10T07:06:37.584 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:06:37.584 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:06:37.584 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:37.584 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:37.584 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:06:37.585 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:06:37.585 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:06:37.585 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:06:37.585 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:06:37.585 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:06:37.585 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2020-04-10T07:06:37.603 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:06:37.603 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rm -f /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:06:37.605 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:2 cluster2 2020-04-10T07:06:37.605 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:2 2020-04-10T07:06:37.605 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:06:37.606 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:37.606 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2020-04-10T07:06:37.606 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:06:37.606 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:06:37.608 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:37.608 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:37.608 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:06:37.608 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:06:37.608 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:06:37.609 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:06:37.609 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:06:37.609 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:06:37.609 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2020-04-10T07:06:37.619 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:06:37.619 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rm -f /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:06:37.622 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_pid_file cluster1:2 2020-04-10T07:06:37.622 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:2 2020-04-10T07:06:37.623 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:37.623 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 cluster instance 2020-04-10T07:06:37.623 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:06:37.623 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:06:37.623 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:37.623 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:37.623 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:06:37.624 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:06:37.624 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:06:37.624 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:37.624 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:06:37.624 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:06:37.624 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 pid file 2020-04-10T07:06:37.635 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.pid 2020-04-10T07:06:37.635 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rm -f /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.pid 2020-04-10T07:06:37.637 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ stop_mirror cluster1:3 2020-04-10T07:06:37.637 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:3 2020-04-10T07:06:37.638 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local sig= 2020-04-10T07:06:37.638 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 2020-04-10T07:06:37.638 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pid 2020-04-10T07:06:37.638 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_pid_file cluster1:3 2020-04-10T07:06:37.638 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:3 2020-04-10T07:06:37.639 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:37.639 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:3 cluster instance 2020-04-10T07:06:37.639 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:3 2020-04-10T07:06:37.639 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:06:37.639 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:37.639 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:37.640 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:06:37.640 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 3 = cluster1:3 ] 2020-04-10T07:06:37.640 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:06:37.640 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:37.640 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=3 2020-04-10T07:06:37.640 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:06:37.640 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.3 pid file 2020-04-10T07:06:37.653 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.pid 2020-04-10T07:06:37.653 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cat /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.pid 2020-04-10T07:06:37.654 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ pid=14714 2020-04-10T07:06:37.654 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n 14714 ] 2020-04-10T07:06:37.654 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ kill 14714 2020-04-10T07:06:37.655 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:06:38.655 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ps auxww 2020-04-10T07:06:38.655 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ awk -v pid=14714 $2 == pid {print; exit 1} 2020-04-10T07:06:38.673 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ break 2020-04-10T07:06:38.673 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ps+ auxww 2020-04-10T07:06:38.673 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:awk -v pid=14714 $2 == pid {print; exit 1} 2020-04-10T07:06:38.690 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:3 cluster1 2020-04-10T07:06:38.690 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:3 2020-04-10T07:06:38.690 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:38.690 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:38.691 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:3 local_cluster instance 2020-04-10T07:06:38.691 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:3 2020-04-10T07:06:38.691 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:06:38.691 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:38.691 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:38.691 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:06:38.691 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 3 = cluster1:3 ] 2020-04-10T07:06:38.692 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:06:38.692 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:06:38.692 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=3 2020-04-10T07:06:38.692 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:06:38.692 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.3 admin socket 2020-04-10T07:06:38.711 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:06:38.711 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rm -f /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:06:38.712 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:3 cluster2 2020-04-10T07:06:38.713 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:3 2020-04-10T07:06:38.713 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:06:38.713 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:38.713 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:3 local_cluster instance 2020-04-10T07:06:38.713 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:3 2020-04-10T07:06:38.713 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:06:38.713 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:38.714 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:38.714 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:06:38.714 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 3 = cluster1:3 ] 2020-04-10T07:06:38.714 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:06:38.714 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:06:38.714 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=3 2020-04-10T07:06:38.714 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:06:38.715 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.3 admin socket 2020-04-10T07:06:38.731 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:06:38.731 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rm -f /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:06:38.733 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_pid_file cluster1:3 2020-04-10T07:06:38.734 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:3 2020-04-10T07:06:38.734 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:38.734 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:3 cluster instance 2020-04-10T07:06:38.734 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:3 2020-04-10T07:06:38.734 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:06:38.734 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:38.734 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:38.735 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:06:38.735 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 3 = cluster1:3 ] 2020-04-10T07:06:38.735 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:06:38.735 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:38.735 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=3 2020-04-10T07:06:38.735 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:06:38.735 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.3 pid file 2020-04-10T07:06:38.748 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.pid 2020-04-10T07:06:38.749 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rm -f /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.pid 2020-04-10T07:06:38.750 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ image1=test1 2020-04-10T07:06:38.750 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ create_image_and_enable_mirror cluster2 mirror test1 2020-04-10T07:06:38.750 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:06:38.750 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:06:38.750 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:38.750 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:06:38.751 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:06:38.751 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:06:38.751 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local mode=snapshot 2020-04-10T07:06:38.751 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:06:38.751 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ create_image cluster2 mirror test1 2020-04-10T07:06:38.751 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:06:38.752 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:06:38.752 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:38.752 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:06:38.752 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:06:38.752 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:06:38.752 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local size=128 2020-04-10T07:06:38.752 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:06:38.753 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 create --size 128 --image-feature layering,exclusive-lock,journaling mirror/test1 2020-04-10T07:06:38.816 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ enable_mirror cluster2 mirror test1 snapshot 2020-04-10T07:06:38.816 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:06:38.816 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:38.816 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:06:38.816 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local mode=snapshot 2020-04-10T07:06:38.816 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/test1 snapshot 2020-04-10T07:06:39.068 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Mirroring enabled 2020-04-10T07:06:39.076 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ write_image cluster2 mirror test1 100 2020-04-10T07:06:39.077 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:06:39.077 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:39.077 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:06:39.077 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local count=100 2020-04-10T07:06:39.077 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local size= 2020-04-10T07:06:39.077 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 2020-04-10T07:06:39.077 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ size=4096 2020-04-10T07:06:39.078 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 bench mirror/test1 --io-type write --io-size 4096 --io-threads 1 --io-total 409600 --io-pattern rand 2020-04-10T07:06:39.147 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:bench type write io_size 4096 io_threads 1 bytes 409600 pattern random 2020-04-10T07:06:39.208 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: SEC OPS OPS/SEC BYTES/SEC 2020-04-10T07:06:39.208 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:elapsed: 0 ops: 100 ops/sec: 1562.5 bytes/sec: 6.1 MiB/s 2020-04-10T07:06:39.220 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ start_mirrors cluster1 2020-04-10T07:06:39.220 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:39.220 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:06:39.220 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ start_mirror cluster1:0 2020-04-10T07:06:39.220 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:06:39.221 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:39.221 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:06:39.221 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:06:39.221 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:06:39.221 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:39.221 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:39.221 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:06:39.222 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:06:39.222 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:06:39.222 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:39.222 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:06:39.222 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:06:39.222 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 2020-04-10T07:06:39.222 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd-mirror --cluster cluster1 --id mirror.0 --rbd-mirror-delete-retry-interval=5 --rbd-mirror-image-state-check-interval=5 --rbd-mirror-journal-poll-age=1 --rbd-mirror-pool-replayers-refresh-interval=5 --debug-rbd=30 --debug-journaler=30 --debug-rbd_mirror=30 --daemonize=true 2020-04-10T07:06:39.246 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ start_mirror cluster1:1 2020-04-10T07:06:39.246 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:1 2020-04-10T07:06:39.246 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:39.246 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 cluster instance 2020-04-10T07:06:39.246 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:06:39.246 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:06:39.247 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:39.247 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:39.247 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:06:39.247 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:06:39.247 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:06:39.247 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:39.247 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:06:39.248 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:06:39.248 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 2020-04-10T07:06:39.248 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd-mirror --cluster cluster1 --id mirror.1 --rbd-mirror-delete-retry-interval=5 --rbd-mirror-image-state-check-interval=5 --rbd-mirror-journal-poll-age=1 --rbd-mirror-pool-replayers-refresh-interval=5 --debug-rbd=30 --debug-journaler=30 --debug-rbd_mirror=30 --daemonize=true 2020-04-10T07:06:39.275 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ start_mirror cluster1:2 2020-04-10T07:06:39.275 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:2 2020-04-10T07:06:39.275 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:39.275 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 cluster instance 2020-04-10T07:06:39.275 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:06:39.276 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:06:39.276 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:39.276 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:39.276 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:06:39.276 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:06:39.276 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:06:39.276 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:39.278 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:06:39.278 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:06:39.278 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 2020-04-10T07:06:39.278 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd-mirror --cluster cluster1 --id mirror.2 --rbd-mirror-delete-retry-interval=5 --rbd-mirror-image-state-check-interval=5 --rbd-mirror-journal-poll-age=1 --rbd-mirror-pool-replayers-refresh-interval=5 --debug-rbd=30 --debug-journaler=30 --debug-rbd_mirror=30 --daemonize=true 2020-04-10T07:06:39.309 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ start_mirror cluster1:3 2020-04-10T07:06:39.310 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:3 2020-04-10T07:06:39.310 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:39.310 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:3 cluster instance 2020-04-10T07:06:39.310 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:3 2020-04-10T07:06:39.310 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:06:39.310 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:39.310 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:39.311 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:06:39.311 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 3 = cluster1:3 ] 2020-04-10T07:06:39.311 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:06:39.311 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:39.311 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=3 2020-04-10T07:06:39.311 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:06:39.311 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 2020-04-10T07:06:39.312 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd-mirror --cluster cluster1 --id mirror.3 --rbd-mirror-delete-retry-interval=5 --rbd-mirror-image-state-check-interval=5 --rbd-mirror-journal-poll-age=1 --rbd-mirror-pool-replayers-refresh-interval=5 --debug-rbd=30 --debug-journaler=30 --debug-rbd_mirror=30 --daemonize=true 2020-04-10T07:06:39.353 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_started cluster1 mirror test1 2020-04-10T07:06:39.353 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:39.353 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:39.353 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:06:39.354 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_state cluster1 mirror test1 started 2020-04-10T07:06:39.354 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:39.354 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:39.354 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:06:39.354 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state=started 2020-04-10T07:06:39.354 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local s 2020-04-10T07:06:39.354 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:06:40.354 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_image_replay_state cluster1 mirror test1 started 2020-04-10T07:06:40.354 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:40.354 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:40.354 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:06:40.355 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local test_state=started 2020-04-10T07:06:40.355 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_result 2020-04-10T07:06:40.355 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_state=stopped 2020-04-10T07:06:40.356 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test1 2020-04-10T07:06:40.356 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_instance=cluster1 2020-04-10T07:06:40.356 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:06:40.357 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ + local cluster=cluster1 2020-04-10T07:06:40.357 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance=cluster1 2020-04-10T07:06:40.357 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local loop_instance 2020-04-10T07:06:40.357 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:grep -i state 2020-04-10T07:06:40.357 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0 2020-04-10T07:06:40.357 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ cluster1 != cluster1 ] 2020-04-10T07:06:40.357 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:06:40.359 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test1 2020-04-10T07:06:40.359 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:06:40.359 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:06:40.359 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:40.359 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:06:40.360 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:06:40.360 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:06:40.360 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:40.360 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:40.360 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:06:40.360 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:06:40.360 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:06:40.361 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:40.361 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:06:40.361 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:06:40.361 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster1 2020-04-10T07:06:40.361 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:06:40.361 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:40.361 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:40.361 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:06:40.362 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:06:40.363 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:06:40.363 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:40.363 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:40.363 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:06:40.363 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:06:40.363 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:06:40.363 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:06:40.364 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:06:40.364 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:06:40.364 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:06:40.378 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:06:40.379 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:06:40.379 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:06:40.379 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test1 2020-04-10T07:06:40.473 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:Invalid command: unused arguments: ['mirror/test1'] 2020-04-10T07:06:40.473 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status : get status for rbd mirror 2020-04-10T07:06:40.473 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:06:40.477 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test1 2020-04-10T07:06:40.477 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:1 2020-04-10T07:06:40.477 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:06:40.477 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:40.478 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 cluster instance 2020-04-10T07:06:40.478 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:06:40.478 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:06:40.478 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:40.478 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:40.479 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:06:40.479 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:06:40.479 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:06:40.480 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:40.480 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:06:40.481 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:06:40.481 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:1 cluster1 2020-04-10T07:06:40.481 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:1 2020-04-10T07:06:40.481 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:40.481 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:40.482 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2020-04-10T07:06:40.482 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:06:40.482 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:06:40.482 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:40.482 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:40.483 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:06:40.483 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:06:40.483 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:06:40.483 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:06:40.483 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:06:40.484 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:06:40.484 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2020-04-10T07:06:40.495 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:06:40.495 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:06:40.496 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:06:40.496 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test1 2020-04-10T07:06:40.584 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:Invalid command: unused arguments: ['mirror/test1'] 2020-04-10T07:06:40.584 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status : get status for rbd mirror 2020-04-10T07:06:40.584 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:06:40.586 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:2 rbd mirror status mirror/test1 2020-04-10T07:06:40.586 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:2 2020-04-10T07:06:40.586 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:06:40.587 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:40.587 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 cluster instance 2020-04-10T07:06:40.587 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:06:40.587 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:06:40.587 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:40.587 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:40.587 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:06:40.587 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:06:40.588 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:06:40.588 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:40.588 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:06:40.588 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:06:40.588 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:2 cluster1 2020-04-10T07:06:40.588 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:2 2020-04-10T07:06:40.588 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:40.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:40.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2020-04-10T07:06:40.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:06:40.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:06:40.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:40.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:40.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:06:40.590 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:06:40.590 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:06:40.590 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:06:40.590 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:06:40.590 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:06:40.590 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2020-04-10T07:06:40.607 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:06:40.608 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:06:40.608 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:06:40.608 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test1 2020-04-10T07:06:40.692 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:Invalid command: unused arguments: ['mirror/test1'] 2020-04-10T07:06:40.693 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status : get status for rbd mirror 2020-04-10T07:06:40.693 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:06:40.697 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:3 rbd mirror status mirror/test1 2020-04-10T07:06:40.697 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:3 2020-04-10T07:06:40.697 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:06:40.698 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:40.698 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:3 cluster instance 2020-04-10T07:06:40.698 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:3 2020-04-10T07:06:40.698 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:06:40.699 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:40.699 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:40.699 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:06:40.699 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 3 = cluster1:3 ] 2020-04-10T07:06:40.699 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:06:40.700 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:40.700 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=3 2020-04-10T07:06:40.700 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:06:40.701 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:3 cluster1 2020-04-10T07:06:40.701 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:3 2020-04-10T07:06:40.701 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:40.701 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:40.702 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:3 local_cluster instance 2020-04-10T07:06:40.702 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:3 2020-04-10T07:06:40.702 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:06:40.703 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:40.704 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:40.704 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:06:40.704 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 3 = cluster1:3 ] 2020-04-10T07:06:40.705 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:06:40.705 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:06:40.705 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=3 2020-04-10T07:06:40.705 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:06:40.705 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.3 admin socket 2020-04-10T07:06:40.717 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:06:40.717 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:06:40.717 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:06:40.717 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/test1 2020-04-10T07:06:40.801 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:Invalid command: unused arguments: ['mirror/test1'] 2020-04-10T07:06:40.802 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status : get status for rbd mirror 2020-04-10T07:06:40.802 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:06:40.802 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:06:41.804 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ cluster1 != cluster1 ] 2020-04-10T07:06:41.804 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:06:41.806 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test1 2020-04-10T07:06:41.807 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:06:41.807 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:06:41.807 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:41.807 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:06:41.807 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:06:41.807 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:06:41.808 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:41.808 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:41.808 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:06:41.808 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:06:41.808 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:06:41.808 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:41.808 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:06:41.809 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:06:41.809 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster1 2020-04-10T07:06:41.809 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:06:41.809 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:41.809 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:41.809 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:06:41.809 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:06:41.810 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:06:41.810 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:41.810 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:41.810 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:06:41.810 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:06:41.810 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:06:41.810 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:06:41.810 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:06:41.811 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:06:41.811 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:06:41.827 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:06:41.827 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:06:41.827 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:06:41.827 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test1 2020-04-10T07:06:41.930 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:06:41.931 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ status_result= "state": "Replaying" 2020-04-10T07:06:41.931 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo "state": "Replaying" 2020-04-10T07:06:41.932 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep -i Replaying 2020-04-10T07:06:41.942 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "state": "Replaying" 2020-04-10T07:06:41.942 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ current_state=started 2020-04-10T07:06:41.942 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test started = started 2020-04-10T07:06:41.942 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:06:41.943 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror test1 2020-04-10T07:06:41.943 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1 2020-04-10T07:06:41.943 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:06:41.943 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:41.943 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:06:41.943 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ snapshot = journal ] 2020-04-10T07:06:41.943 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ snapshot = snapshot ] 2020-04-10T07:06:41.944 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror test1 2020-04-10T07:06:41.944 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1 2020-04-10T07:06:41.944 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:06:41.944 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:41.944 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:06:41.944 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster2-mirror-test1.status 2020-04-10T07:06:41.944 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2-mirror-test1.status 2020-04-10T07:06:41.944 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:06:41.945 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster2-mirror-test1.status 2020-04-10T07:06:41.945 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror-test1.status 2020-04-10T07:06:41.945 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror-test1.status 2020-04-10T07:06:41.945 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:06:41.945 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.status 2020-04-10T07:06:41.945 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mirror_image_snapshot cluster2 mirror test1 2020-04-10T07:06:41.945 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:06:41.946 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:41.946 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:06:41.946 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test1 2020-04-10T07:06:42.086 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Snapshot ID: 8 2020-04-10T07:06:42.096 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_newest_mirror_snapshot cluster2 mirror test1 /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test1.status 2020-04-10T07:06:42.096 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:06:42.097 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:42.097 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:06:42.097 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local log=/tmp/tmp.8SwBCsYPSC/cluster2-mirror-test1.status 2020-04-10T07:06:42.097 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 snap list --all mirror/test1 --format xml 2020-04-10T07:06:42.097 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2020-04-10T07:06:42.147 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //snapshot/id 2020-04-10T07:06:42.150 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snapshot_id=8 2020-04-10T07:06:42.151 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ true 2020-04-10T07:06:42.151 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0.2 2020-04-10T07:06:42.352 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_newest_mirror_snapshot cluster1 mirror test1 /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.status 2020-04-10T07:06:42.352 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:42.353 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:42.353 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:06:42.353 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.status 2020-04-10T07:06:42.353 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 snap list --all mirror/test1 --format xml 2020-04-10T07:06:42.353 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2020-04-10T07:06:42.420 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2020-04-10T07:06:42.425 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local primary_snapshot_id=7 2020-04-10T07:06:42.425 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test 8 = 7 2020-04-10T07:06:42.425 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0.4 2020-04-10T07:06:42.827 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_newest_mirror_snapshot cluster1 mirror test1 /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.status 2020-04-10T07:06:42.827 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:42.827 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:42.827 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:06:42.828 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.status 2020-04-10T07:06:42.828 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 snap list --all mirror/test1 --format xml 2020-04-10T07:06:42.828 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2020-04-10T07:06:42.895 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2020-04-10T07:06:42.899 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local primary_snapshot_id=7 2020-04-10T07:06:42.899 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test 8 = 7 2020-04-10T07:06:42.899 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0.8 2020-04-10T07:06:43.701 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_newest_mirror_snapshot cluster1 mirror test1 /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.status 2020-04-10T07:06:43.701 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:43.701 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:43.701 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:06:43.702 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.status 2020-04-10T07:06:43.702 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 snap list --all mirror/test1 --format xml 2020-04-10T07:06:43.702 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2020-04-10T07:06:43.768 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2020-04-10T07:06:43.774 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local primary_snapshot_id=8 2020-04-10T07:06:43.774 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test 8 = 8 2020-04-10T07:06:43.774 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:06:43.775 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test1 up+replaying 2020-04-10T07:06:43.775 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:43.775 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:43.775 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:06:43.775 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:06:43.775 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:06:43.775 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:06:43.776 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:06:44.776 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_status_in_pool_dir cluster1 mirror test1 up+replaying 2020-04-10T07:06:44.776 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:44.776 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:44.776 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:06:44.776 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:06:44.777 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:06:44.777 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:06:44.777 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror-test1.mirror_status 2020-04-10T07:06:44.777 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror-test1.mirror_status 2020-04-10T07:06:44.777 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:06:44.778 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.mirror_status 2020-04-10T07:06:44.778 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test1 2020-04-10T07:06:44.779 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ tee /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.mirror_status 2020-04-10T07:06:44.844 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:test1: 2020-04-10T07:06:44.844 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: global_id: 56419286-13e9-49f6-a20d-2abe6ede083e 2020-04-10T07:06:44.844 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+replaying 2020-04-10T07:06:44.845 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: replaying, {"replay_state": "idle", "remote_snapshot_timestamp": 1586502399} 2020-04-10T07:06:44.845 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: service: mirror.0 on smithi204 2020-04-10T07:06:44.845 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:06:41 2020-04-10T07:06:44.853 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ state: .*up+replaying /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.mirror_status 2020-04-10T07:06:44.853 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+replaying 2020-04-10T07:06:44.853 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ description: .* /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.mirror_status 2020-04-10T07:06:44.855 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: replaying, {"replay_state": "idle", "remote_snapshot_timestamp": 1586502399} 2020-04-10T07:06:44.855 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:06:44.856 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+replaying 2020-04-10T07:06:44.856 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^up+ 2020-04-10T07:06:44.857 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+replaying 2020-04-10T07:06:44.857 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep service: *mirror..* on /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.mirror_status 2020-04-10T07:06:44.859 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: service: mirror.0 on smithi204 2020-04-10T07:06:44.859 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.mirror_status 2020-04-10T07:06:44.861 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update=2020-04-10 07:06:41 2020-04-10T07:06:44.861 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test1 up+replaying 2020-04-10 07:06:41 2020-04-10T07:06:44.862 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:44.862 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:44.862 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:06:44.862 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:06:44.862 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local prev_last_update=2020-04-10 07:06:41 2020-04-10T07:06:44.862 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror.mirror_status 2020-04-10T07:06:44.863 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror.mirror_status 2020-04-10T07:06:44.863 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:06:44.864 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror.mirror_status 2020-04-10T07:06:44.864 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2020-04-10T07:06:44.906 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: failed to query services: (13) Permission denied 2020-04-10T07:06:44.953 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update state 2020-04-10T07:06:44.954 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test1']/last_update 2020-04-10T07:06:44.959 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ last_update=2020-04-10 07:06:41 2020-04-10T07:06:44.959 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test1']/state 2020-04-10T07:06:44.965 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ state=up+replaying 2020-04-10T07:06:44.965 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+replaying 2020-04-10T07:06:44.965 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep up+replaying 2020-04-10T07:06:44.966 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+replaying 2020-04-10T07:06:44.967 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:06:44.967 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:06:44.968 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -z ] 2020-04-10T07:06:44.968 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_status_in_pool_dir cluster2 mirror test1 down+unknown 2020-04-10T07:06:44.968 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:06:44.968 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:44.968 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:06:44.968 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=down+unknown 2020-04-10T07:06:44.969 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:06:44.969 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:06:44.969 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:06:45.968 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_status_in_pool_dir cluster2 mirror test1 down+unknown 2020-04-10T07:06:45.969 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:06:45.969 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:45.969 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:06:45.969 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=down+unknown 2020-04-10T07:06:45.969 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:06:45.969 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:06:45.969 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster2-mirror-test1.mirror_status 2020-04-10T07:06:45.970 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2-mirror-test1.mirror_status 2020-04-10T07:06:45.970 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:06:45.971 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster2-mirror-test1.mirror_status 2020-04-10T07:06:45.971 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster2 mirror image status mirror/test1 2020-04-10T07:06:45.971 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ tee /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test1.mirror_status 2020-04-10T07:06:46.029 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:test1: 2020-04-10T07:06:46.029 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: global_id: 56419286-13e9-49f6-a20d-2abe6ede083e 2020-04-10T07:06:46.029 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: down+unknown 2020-04-10T07:06:46.029 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: status not found 2020-04-10T07:06:46.030 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10T07:06:46.030 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: peer_sites: 2020-04-10T07:06:46.030 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: name: cluster1-DNE 2020-04-10T07:06:46.030 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+replaying 2020-04-10T07:06:46.030 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: replaying, {"replay_state": "idle", "remote_snapshot_timestamp": 1586502399} 2020-04-10T07:06:46.030 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:06:41 2020-04-10T07:06:46.030 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: snapshots: 2020-04-10T07:06:46.031 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: 8 .mirror.primary.56419286-13e9-49f6-a20d-2abe6ede083e.86485e40-03ca-42ca-b955-dd92737e5c6e (peer_uuids:[90b6cfa2-5481-4edf-b462-5494aab62f6f]) 2020-04-10T07:06:46.034 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ state: .*down+unknown /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test1.mirror_status 2020-04-10T07:06:46.035 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: down+unknown 2020-04-10T07:06:46.035 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ description: .* /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test1.mirror_status 2020-04-10T07:06:46.036 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: status not found 2020-04-10T07:06:46.037 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:06:46.037 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo down+unknown 2020-04-10T07:06:46.037 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^up+ 2020-04-10T07:06:46.037 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep service: /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test1.mirror_status 2020-04-10T07:06:46.038 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test1.mirror_status 2020-04-10T07:06:46.041 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update= 2020-04-10T07:06:46.041 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_mirror_pool_status_verbose cluster2 mirror test1 down+unknown 2020-04-10T07:06:46.041 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:06:46.041 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:46.041 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:06:46.041 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=down+unknown 2020-04-10T07:06:46.041 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local prev_last_update= 2020-04-10T07:06:46.042 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster2-mirror.mirror_status 2020-04-10T07:06:46.042 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2-mirror.mirror_status 2020-04-10T07:06:46.042 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:06:46.042 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster2-mirror.mirror_status 2020-04-10T07:06:46.043 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2020-04-10T07:06:46.078 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: failed to query services: (13) Permission denied 2020-04-10T07:06:46.112 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update state 2020-04-10T07:06:46.112 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test1']/last_update 2020-04-10T07:06:46.115 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ last_update= 2020-04-10T07:06:46.115 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test1']/state 2020-04-10T07:06:46.117 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ state=down+unknown 2020-04-10T07:06:46.117 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo down+unknown 2020-04-10T07:06:46.118 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep down+unknown 2020-04-10T07:06:46.119 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:down+unknown 2020-04-10T07:06:46.119 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:06:46.119 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:06:46.119 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ compare_images mirror test1 2020-04-10T07:06:46.119 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:46.120 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:06:46.120 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local ret=0 2020-04-10T07:06:46.120 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster2-mirror-test1.export 2020-04-10T07:06:46.120 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2-mirror-test1.export 2020-04-10T07:06:46.120 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:06:46.121 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local rmt_export=/tmp/tmp.8SwBCsYPSC/cluster2-mirror-test1.export 2020-04-10T07:06:46.121 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror-test1.export 2020-04-10T07:06:46.122 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror-test1.export 2020-04-10T07:06:46.122 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:06:46.122 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local loc_export=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.export 2020-04-10T07:06:46.122 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rm -f /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test1.export /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.export 2020-04-10T07:06:46.122 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 export mirror/test1 /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test1.export 2020-04-10T07:06:46.289 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2020-04-10T07:06:46.292 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 export mirror/test1 /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.export 2020-04-10T07:06:46.499 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2020-04-10T07:06:46.502 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cmp /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test1.export /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.export 2020-04-10T07:06:46.577 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rm -f /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test1.export /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.export 2020-04-10T07:06:46.613 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:06:46.613 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ testlog TEST: test the first image is replaying after restart 2020-04-10T07:06:46.614 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ tee -a /tmp/tmp.8SwBCsYPSC/rbd-mirror.test.log 2020-04-10T07:06:46.614 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ date +%F %T 2020-04-10T07:06:46.615 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10 07:06:46 TEST: test the first image is replaying after restart 2020-04-10T07:06:46.615 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:2020-04-10 07:06:46 TEST: test the first image is replaying after restart 2020-04-10T07:06:46.615 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ write_image cluster2 mirror test 100 2020-04-10T07:06:46.615 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:06:46.615 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:46.615 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:06:46.616 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local count=100 2020-04-10T07:06:46.616 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local size= 2020-04-10T07:06:46.616 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 2020-04-10T07:06:46.616 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ size=4096 2020-04-10T07:06:46.616 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 bench mirror/test --io-type write --io-size 4096 --io-threads 1 --io-total 409600 --io-pattern rand 2020-04-10T07:06:46.696 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:bench type write io_size 4096 io_threads 1 bytes 409600 pattern random 2020-04-10T07:06:46.760 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: SEC OPS OPS/SEC BYTES/SEC 2020-04-10T07:06:46.760 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:elapsed: 0 ops: 100 ops/sec: 1562.5 bytes/sec: 6.1 MiB/s 2020-04-10T07:06:46.771 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_started cluster1 mirror test 2020-04-10T07:06:46.771 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:46.771 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:46.772 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:06:46.772 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2020-04-10T07:06:46.772 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:46.772 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:46.772 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:06:46.772 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state=started 2020-04-10T07:06:46.772 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local s 2020-04-10T07:06:46.773 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:06:47.772 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_image_replay_state cluster1 mirror test started 2020-04-10T07:06:47.772 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:47.772 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:47.772 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:06:47.773 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local test_state=started 2020-04-10T07:06:47.773 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_result 2020-04-10T07:06:47.773 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_state=stopped 2020-04-10T07:06:47.774 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2020-04-10T07:06:47.774 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ + local cluster_instance=cluster1 2020-04-10T07:06:47.774 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shiftgrep 2020-04-10T07:06:47.774 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: -i state 2020-04-10T07:06:47.775 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:47.775 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance=cluster1 2020-04-10T07:06:47.775 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local loop_instance 2020-04-10T07:06:47.775 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0 2020-04-10T07:06:47.775 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ cluster1 != cluster1 ] 2020-04-10T07:06:47.775 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:06:47.777 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2020-04-10T07:06:47.777 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:06:47.777 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:06:47.777 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:47.778 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:06:47.778 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:06:47.778 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:06:47.778 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:47.778 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:47.778 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:06:47.778 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:06:47.779 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:06:47.779 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:47.779 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:06:47.779 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:06:47.779 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster1 2020-04-10T07:06:47.779 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:06:47.779 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:47.779 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:47.780 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:06:47.780 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:06:47.780 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:06:47.780 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:47.780 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:47.780 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:06:47.780 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:06:47.781 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:06:47.781 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:06:47.782 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:06:47.782 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:06:47.782 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:06:47.800 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:06:47.800 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:06:47.800 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:06:47.800 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2020-04-10T07:06:47.896 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:no valid command found; 1 closest matches: 2020-04-10T07:06:47.896 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status mirror/test1 2020-04-10T07:06:47.897 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:06:47.898 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2020-04-10T07:06:47.899 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:1 2020-04-10T07:06:47.899 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:06:47.899 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:47.899 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 cluster instance 2020-04-10T07:06:47.899 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:06:47.899 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:06:47.899 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:47.900 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:47.900 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:06:47.900 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:06:47.900 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:06:47.900 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:47.900 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:06:47.900 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:06:47.901 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:1 cluster1 2020-04-10T07:06:47.901 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:1 2020-04-10T07:06:47.901 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:47.901 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:47.901 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2020-04-10T07:06:47.901 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:06:47.901 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:06:47.902 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:47.902 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:47.902 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:06:47.902 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:06:47.902 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:06:47.902 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:06:47.902 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:06:47.902 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:06:47.903 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2020-04-10T07:06:47.919 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:06:47.919 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:06:47.919 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:06:47.919 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2020-04-10T07:06:48.019 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:06:48.020 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ status_result= "state": "Replaying" 2020-04-10T07:06:48.020 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep -i Replaying 2020-04-10T07:06:48.020 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo "state": "Replaying" 2020-04-10T07:06:48.022 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "state": "Replaying" 2020-04-10T07:06:48.022 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ current_state=started 2020-04-10T07:06:48.023 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test started = started 2020-04-10T07:06:48.023 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:06:48.023 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror test 2020-04-10T07:06:48.023 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1 2020-04-10T07:06:48.023 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:06:48.023 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:48.023 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:06:48.023 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ snapshot = journal ] 2020-04-10T07:06:48.024 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ snapshot = snapshot ] 2020-04-10T07:06:48.024 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror test 2020-04-10T07:06:48.024 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1 2020-04-10T07:06:48.024 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:06:48.024 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:48.024 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:06:48.024 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster2-mirror-test.status 2020-04-10T07:06:48.025 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2-mirror-test.status 2020-04-10T07:06:48.025 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:06:48.026 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.status 2020-04-10T07:06:48.026 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror-test.status 2020-04-10T07:06:48.026 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror-test.status 2020-04-10T07:06:48.026 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:06:48.026 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.status 2020-04-10T07:06:48.027 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mirror_image_snapshot cluster2 mirror test 2020-04-10T07:06:48.027 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:06:48.027 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:48.027 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:06:48.027 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test 2020-04-10T07:06:49.110 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Snapshot ID: 10 2020-04-10T07:06:49.121 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_newest_mirror_snapshot cluster2 mirror test /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.status 2020-04-10T07:06:49.121 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:06:49.121 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:49.121 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:06:49.121 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local log=/tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.status 2020-04-10T07:06:49.122 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 snap list --all mirror/test --format xml 2020-04-10T07:06:49.122 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2020-04-10T07:06:49.169 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //snapshot/id 2020-04-10T07:06:49.172 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snapshot_id=10 2020-04-10T07:06:49.173 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ true 2020-04-10T07:06:49.173 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0.2 2020-04-10T07:06:49.374 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.status 2020-04-10T07:06:49.375 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:49.375 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:49.375 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:06:49.375 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.status 2020-04-10T07:06:49.375 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2020-04-10T07:06:49.375 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2020-04-10T07:06:49.441 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2020-04-10T07:06:49.447 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local primary_snapshot_id=5 2020-04-10T07:06:49.447 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test 10 = 5 2020-04-10T07:06:49.448 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0.4 2020-04-10T07:06:49.850 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.status 2020-04-10T07:06:49.850 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:49.850 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:49.850 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:06:49.850 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.status 2020-04-10T07:06:49.851 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2020-04-10T07:06:49.851 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2020-04-10T07:06:49.918 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2020-04-10T07:06:49.924 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local primary_snapshot_id=10 2020-04-10T07:06:49.924 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test 10 = 10 2020-04-10T07:06:49.924 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:06:49.924 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2020-04-10T07:06:49.924 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:49.925 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:49.925 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:06:49.925 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:06:49.925 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:06:49.925 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:06:49.925 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:06:50.925 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying 2020-04-10T07:06:50.925 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:50.926 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:50.926 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:06:50.926 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:06:50.926 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:06:50.926 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:06:50.926 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror-test.mirror_status 2020-04-10T07:06:50.927 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror-test.mirror_status 2020-04-10T07:06:50.927 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:06:50.928 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:06:50.928 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status+ mirror/test 2020-04-10T07:06:50.928 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:tee /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:06:51.004 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:test: 2020-04-10T07:06:51.004 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: global_id: 687e9f7d-ae46-4324-b8f7-7c16605e0fa6 2020-04-10T07:06:51.004 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+replaying 2020-04-10T07:06:51.005 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: replaying, {"replay_state": "idle", "remote_snapshot_timestamp": 1586502390, "local_snapshot_timestamp": 1586502390} 2020-04-10T07:06:51.005 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: service: mirror.1 on smithi204 2020-04-10T07:06:51.005 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:06:43 2020-04-10T07:06:51.012 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ state: .*up+replaying /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:06:51.013 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+replaying 2020-04-10T07:06:51.014 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ description: .* /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:06:51.016 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: replaying, {"replay_state": "idle", "remote_snapshot_timestamp": 1586502390, "local_snapshot_timestamp": 1586502390} 2020-04-10T07:06:51.016 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:06:51.016 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+replaying 2020-04-10T07:06:51.017 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^up+ 2020-04-10T07:06:51.017 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+replaying 2020-04-10T07:06:51.018 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep service: *mirror..* on /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:06:51.019 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: service: mirror.1 on smithi204 2020-04-10T07:06:51.020 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:06:51.022 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update=2020-04-10 07:06:43 2020-04-10T07:06:51.022 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying 2020-04-10 07:06:43 2020-04-10T07:06:51.022 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:51.022 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:51.022 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:06:51.022 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:06:51.022 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local prev_last_update=2020-04-10 07:06:43 2020-04-10T07:06:51.023 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror.mirror_status 2020-04-10T07:06:51.023 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror.mirror_status 2020-04-10T07:06:51.023 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:06:51.024 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror.mirror_status 2020-04-10T07:06:51.024 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2020-04-10T07:06:51.067 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: failed to query services: (13) Permission denied 2020-04-10T07:06:51.113 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update state 2020-04-10T07:06:51.114 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2020-04-10T07:06:51.114 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ last_update=2020-04-10 07:06:43 2020-04-10T07:06:51.114 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2020-04-10T07:06:51.116 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ state=up+replaying 2020-04-10T07:06:51.116 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo+ up+replaying 2020-04-10T07:06:51.116 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:grep up+replaying 2020-04-10T07:06:51.116 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:06:51.117 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:06:51.117 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ compare_images mirror test 2020-04-10T07:06:51.117 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:51.117 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:06:51.117 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local ret=0 2020-04-10T07:06:51.117 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster2-mirror-test.export 2020-04-10T07:06:51.117 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2-mirror-test.export 2020-04-10T07:06:51.118 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+replaying 2020-04-10T07:06:51.119 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:06:51.119 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local rmt_export=/tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.export 2020-04-10T07:06:51.119 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror-test.export 2020-04-10T07:06:51.119 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror-test.export 2020-04-10T07:06:51.119 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:06:51.120 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local loc_export=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.export 2020-04-10T07:06:51.120 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rm -f /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.export /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.export 2020-04-10T07:06:51.120 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 export mirror/test /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.export 2020-04-10T07:06:51.300 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2020-04-10T07:06:51.305 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 export mirror/test /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.export 2020-04-10T07:06:51.479 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2020-04-10T07:06:51.483 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cmp /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.export /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.export 2020-04-10T07:06:51.557 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rm -f /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.export /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.export 2020-04-10T07:06:51.596 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:06:51.596 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -z ] 2020-04-10T07:06:51.596 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ testlog TEST: stop/start/restart mirror via admin socket 2020-04-10T07:06:51.597 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ tee -a /tmp/tmp.8SwBCsYPSC/rbd-mirror.test.log 2020-04-10T07:06:51.597 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ date +%F %T 2020-04-10T07:06:51.602 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10 07:06:51 TEST: stop/start/restart mirror via admin socket 2020-04-10T07:06:51.602 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:2020-04-10 07:06:51 TEST: stop/start/restart mirror via admin socket 2020-04-10T07:06:51.602 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ all_admin_daemons cluster1 rbd mirror stop 2020-04-10T07:06:51.603 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:51.603 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:06:51.603 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:06:51.603 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:0 rbd mirror stop 2020-04-10T07:06:51.603 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:06:51.603 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:06:51.603 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:51.604 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:06:51.604 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:06:51.604 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:06:51.604 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:51.604 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:51.604 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:06:51.604 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:06:51.605 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:06:51.605 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:51.605 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:06:51.605 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:06:51.605 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster1 2020-04-10T07:06:51.605 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:06:51.605 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:51.606 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:51.606 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:06:51.606 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:06:51.606 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:06:51.606 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:51.606 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:51.606 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:06:51.606 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:06:51.607 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:06:51.607 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:06:51.607 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:06:51.607 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:06:51.607 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:06:51.615 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:06:51.615 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:06:51.616 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:06:51.616 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror stop 2020-04-10T07:06:51.715 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:1 rbd mirror stop 2020-04-10T07:06:51.715 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:1 2020-04-10T07:06:51.715 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:06:51.716 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:51.716 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 cluster instance 2020-04-10T07:06:51.716 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:06:51.716 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:06:51.716 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:51.716 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:51.716 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:06:51.717 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:06:51.717 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:06:51.717 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:51.717 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:06:51.717 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:06:51.717 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:1 cluster1 2020-04-10T07:06:51.717 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:1 2020-04-10T07:06:51.718 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:51.718 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:51.718 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2020-04-10T07:06:51.718 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:06:51.718 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:06:51.718 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:51.718 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:51.718 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:06:51.719 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:06:51.719 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:06:51.719 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:06:51.719 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:06:51.719 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:06:51.719 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2020-04-10T07:06:51.729 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:06:51.729 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:06:51.729 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:06:51.729 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror stop 2020-04-10T07:06:51.822 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:2 rbd mirror stop 2020-04-10T07:06:51.822 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:2 2020-04-10T07:06:51.823 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:06:51.823 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:51.823 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 cluster instance 2020-04-10T07:06:51.823 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:06:51.823 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:06:51.823 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:51.823 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:51.824 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:06:51.824 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:06:51.824 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:06:51.824 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:51.824 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:06:51.825 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:06:51.825 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:2 cluster1 2020-04-10T07:06:51.825 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:2 2020-04-10T07:06:51.825 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:51.825 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:51.825 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2020-04-10T07:06:51.825 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:06:51.826 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:06:51.826 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:51.826 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:51.827 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:06:51.827 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:06:51.827 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:06:51.827 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:06:51.827 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:06:51.827 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:06:51.827 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2020-04-10T07:06:51.843 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:06:51.843 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:06:51.843 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:06:51.843 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror stop 2020-04-10T07:06:51.934 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:3 rbd mirror stop 2020-04-10T07:06:51.934 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:3 2020-04-10T07:06:51.935 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:06:51.935 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:51.935 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:3 cluster instance 2020-04-10T07:06:51.935 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:3 2020-04-10T07:06:51.935 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:06:51.935 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:51.936 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:51.936 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:06:51.936 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 3 = cluster1:3 ] 2020-04-10T07:06:51.936 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:06:51.936 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:51.936 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=3 2020-04-10T07:06:51.939 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:06:51.939 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:3 cluster1 2020-04-10T07:06:51.939 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:3 2020-04-10T07:06:51.939 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:51.939 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:51.940 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:3 local_cluster instance 2020-04-10T07:06:51.940 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:3 2020-04-10T07:06:51.940 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:06:51.940 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:51.940 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:51.940 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:06:51.940 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 3 = cluster1:3 ] 2020-04-10T07:06:51.941 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:06:51.941 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:06:51.941 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=3 2020-04-10T07:06:51.941 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:06:51.941 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.3 admin socket 2020-04-10T07:06:51.953 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:06:51.954 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:06:51.954 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:06:51.954 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror stop 2020-04-10T07:06:52.052 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_stopped cluster1 mirror test 2020-04-10T07:06:52.052 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:52.052 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:52.052 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:06:52.052 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_state cluster1 mirror test stopped 2020-04-10T07:06:52.053 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:52.053 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:52.053 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:06:52.053 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state=stopped 2020-04-10T07:06:52.053 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local s 2020-04-10T07:06:52.053 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:06:53.052 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_image_replay_state cluster1 mirror test stopped 2020-04-10T07:06:53.052 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:53.053 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:53.053 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:06:53.053 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local test_state=stopped 2020-04-10T07:06:53.053 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_result 2020-04-10T07:06:53.053 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_state=stopped 2020-04-10T07:06:53.054 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2020-04-10T07:06:53.055 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local+ cluster_instance=cluster1 2020-04-10T07:06:53.055 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:grep -i+ state 2020-04-10T07:06:53.055 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:shift 2020-04-10T07:06:53.055 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:53.055 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance=cluster1 2020-04-10T07:06:53.055 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local loop_instance 2020-04-10T07:06:53.055 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0 2020-04-10T07:06:53.056 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ cluster1 != cluster1 ] 2020-04-10T07:06:53.056 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:06:53.057 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2020-04-10T07:06:53.057 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:06:53.057 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:06:53.057 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:53.058 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:06:53.058 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:06:53.058 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:06:53.058 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:53.058 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:53.058 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:06:53.058 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:06:53.059 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:06:53.059 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:53.059 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:06:53.059 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:06:53.059 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster1 2020-04-10T07:06:53.059 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:06:53.059 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:53.059 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:53.061 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:06:53.061 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:06:53.061 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:06:53.061 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:53.061 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:53.061 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:06:53.062 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:06:53.062 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:06:53.062 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:06:53.062 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:06:53.062 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:06:53.062 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:06:53.078 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:06:53.078 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:06:53.078 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:06:53.079 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2020-04-10T07:06:53.173 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:no valid command found; 1 closest matches: 2020-04-10T07:06:53.173 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status mirror/test1 2020-04-10T07:06:53.174 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:06:53.175 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2020-04-10T07:06:53.176 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:1 2020-04-10T07:06:53.176 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:06:53.176 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:53.176 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 cluster instance 2020-04-10T07:06:53.176 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:06:53.176 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:06:53.177 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:53.177 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:53.177 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:06:53.177 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:06:53.177 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:06:53.177 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:53.177 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:06:53.178 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:06:53.178 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:1 cluster1 2020-04-10T07:06:53.178 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:1 2020-04-10T07:06:53.178 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:53.178 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:53.178 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2020-04-10T07:06:53.178 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:06:53.178 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:06:53.179 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:53.179 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:53.179 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:06:53.179 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:06:53.179 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:06:53.179 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:06:53.179 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:06:53.180 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:06:53.180 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2020-04-10T07:06:53.195 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:06:53.195 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:06:53.195 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:06:53.196 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2020-04-10T07:06:53.290 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:06:53.291 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ status_result= "state": "Stopped" 2020-04-10T07:06:53.291 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo "state": "Stopped" 2020-04-10T07:06:53.291 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep -i Replaying 2020-04-10T07:06:53.294 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test stopped = stopped 2020-04-10T07:06:53.294 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:06:53.294 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_stopped cluster1 mirror test1 2020-04-10T07:06:53.294 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:53.294 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:53.295 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:06:53.295 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_state cluster1 mirror test1 stopped 2020-04-10T07:06:53.295 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:53.295 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:53.295 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:06:53.295 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state=stopped 2020-04-10T07:06:53.295 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local s 2020-04-10T07:06:53.296 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:06:54.294 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_image_replay_state cluster1 mirror test1 stopped 2020-04-10T07:06:54.294 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:54.294 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:54.294 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:06:54.295 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local test_state=stopped 2020-04-10T07:06:54.295 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_result 2020-04-10T07:06:54.295 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_state=stopped 2020-04-10T07:06:54.296 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test1 2020-04-10T07:06:54.296 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ + local cluster_instance=cluster1 2020-04-10T07:06:54.296 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:06:54.296 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:grep -i state 2020-04-10T07:06:54.297 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:54.297 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance=cluster1 2020-04-10T07:06:54.297 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local loop_instance 2020-04-10T07:06:54.297 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0 2020-04-10T07:06:54.297 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ cluster1 != cluster1 ] 2020-04-10T07:06:54.297 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:06:54.299 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test1 2020-04-10T07:06:54.299 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:06:54.299 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:06:54.299 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:54.300 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:06:54.300 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:06:54.300 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:06:54.300 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:54.300 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:54.300 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:06:54.300 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:06:54.301 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:06:54.301 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:54.301 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:06:54.301 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:06:54.301 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster1 2020-04-10T07:06:54.301 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:06:54.301 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:54.302 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:54.302 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:06:54.302 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:06:54.302 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:06:54.303 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:54.303 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:54.303 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:06:54.303 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:06:54.303 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:06:54.304 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:06:54.304 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:06:54.304 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:06:54.304 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:06:54.319 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:06:54.319 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:06:54.319 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:06:54.319 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test1 2020-04-10T07:06:54.425 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:06:54.426 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ status_result= "state": "Stopped" 2020-04-10T07:06:54.426 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo "state": "Stopped" 2020-04-10T07:06:54.427 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep -i Replaying 2020-04-10T07:06:54.429 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test stopped = stopped 2020-04-10T07:06:54.429 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:06:54.429 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+stopped 2020-04-10T07:06:54.429 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:54.429 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:54.430 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:06:54.430 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+stopped 2020-04-10T07:06:54.430 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:06:54.430 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:06:54.430 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:06:55.430 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_status_in_pool_dir cluster1 mirror test up+stopped 2020-04-10T07:06:55.431 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:55.431 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:55.431 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:06:55.431 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+stopped 2020-04-10T07:06:55.431 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:06:55.431 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:06:55.431 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror-test.mirror_status 2020-04-10T07:06:55.431 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror-test.mirror_status 2020-04-10T07:06:55.432 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:06:55.432 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:06:55.433 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test 2020-04-10T07:06:55.433 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ tee /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:06:55.504 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:test: 2020-04-10T07:06:55.504 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: global_id: 687e9f7d-ae46-4324-b8f7-7c16605e0fa6 2020-04-10T07:06:55.505 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+stopped 2020-04-10T07:06:55.505 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: stopped 2020-04-10T07:06:55.505 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: service: mirror.1 on smithi204 2020-04-10T07:06:55.505 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:06:51 2020-04-10T07:06:55.512 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ state: .*up+stopped /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:06:55.515 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+stopped 2020-04-10T07:06:55.516 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ description: .* /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:06:55.517 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: stopped 2020-04-10T07:06:55.517 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:06:55.517 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+stopped 2020-04-10T07:06:55.517 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^up+ 2020-04-10T07:06:55.518 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+stopped 2020-04-10T07:06:55.518 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep service: *mirror..* on /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:06:55.519 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: service: mirror.1 on smithi204 2020-04-10T07:06:55.520 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:06:55.522 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update=2020-04-10 07:06:51 2020-04-10T07:06:55.522 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+stopped 2020-04-10 07:06:51 2020-04-10T07:06:55.522 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:55.523 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:55.523 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:06:55.523 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+stopped 2020-04-10T07:06:55.523 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local prev_last_update=2020-04-10 07:06:51 2020-04-10T07:06:55.523 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror.mirror_status 2020-04-10T07:06:55.525 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror.mirror_status 2020-04-10T07:06:55.525 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:06:55.525 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror.mirror_status 2020-04-10T07:06:55.525 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2020-04-10T07:06:55.568 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: failed to query services: (13) Permission denied 2020-04-10T07:06:55.617 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update state 2020-04-10T07:06:55.617 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2020-04-10T07:06:55.623 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ last_update=2020-04-10 07:06:51 2020-04-10T07:06:55.623 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2020-04-10T07:06:55.628 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ state=up+stopped 2020-04-10T07:06:55.629 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+stopped 2020-04-10T07:06:55.629 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep up+stopped 2020-04-10T07:06:55.631 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+stopped 2020-04-10T07:06:55.631 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:06:55.632 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:06:55.632 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test1 up+stopped 2020-04-10T07:06:55.632 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:55.632 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:55.633 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:06:55.633 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+stopped 2020-04-10T07:06:55.633 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:06:55.633 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:06:55.633 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:06:56.632 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_status_in_pool_dir cluster1 mirror test1 up+stopped 2020-04-10T07:06:56.632 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:56.633 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:56.633 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:06:56.633 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+stopped 2020-04-10T07:06:56.633 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:06:56.633 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:06:56.633 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror-test1.mirror_status 2020-04-10T07:06:56.633 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror-test1.mirror_status 2020-04-10T07:06:56.634 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:06:56.637 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.mirror_status 2020-04-10T07:06:56.637 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test1 2020-04-10T07:06:56.637 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ tee /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.mirror_status 2020-04-10T07:06:56.701 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:test1: 2020-04-10T07:06:56.701 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: global_id: 56419286-13e9-49f6-a20d-2abe6ede083e 2020-04-10T07:06:56.701 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+stopped 2020-04-10T07:06:56.702 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: stopped 2020-04-10T07:06:56.702 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: service: mirror.0 on smithi204 2020-04-10T07:06:56.702 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:06:51 2020-04-10T07:06:56.709 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ state: .*up+stopped /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.mirror_status 2020-04-10T07:06:56.711 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+stopped 2020-04-10T07:06:56.711 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ description: .* /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.mirror_status 2020-04-10T07:06:56.712 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: stopped 2020-04-10T07:06:56.713 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:06:56.713 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+stopped 2020-04-10T07:06:56.713 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^up+ 2020-04-10T07:06:56.714 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+stopped 2020-04-10T07:06:56.714 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep service: *mirror..* on /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.mirror_status 2020-04-10T07:06:56.716 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: service: mirror.0 on smithi204 2020-04-10T07:06:56.717 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.mirror_status 2020-04-10T07:06:56.719 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update=2020-04-10 07:06:51 2020-04-10T07:06:56.719 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test1 up+stopped 2020-04-10 07:06:51 2020-04-10T07:06:56.719 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:56.719 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:56.719 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:06:56.719 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+stopped 2020-04-10T07:06:56.719 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local prev_last_update=2020-04-10 07:06:51 2020-04-10T07:06:56.720 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror.mirror_status 2020-04-10T07:06:56.720 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror.mirror_status 2020-04-10T07:06:56.720 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:06:56.721 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror.mirror_status 2020-04-10T07:06:56.721 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2020-04-10T07:06:56.764 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: failed to query services: (13) Permission denied 2020-04-10T07:06:56.812 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update state 2020-04-10T07:06:56.812 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test1']/last_update 2020-04-10T07:06:56.818 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ last_update=2020-04-10 07:06:51 2020-04-10T07:06:56.818 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test1']/state 2020-04-10T07:06:56.822 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ state=up+stopped 2020-04-10T07:06:56.823 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+stopped 2020-04-10T07:06:56.823 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep up+stopped 2020-04-10T07:06:56.824 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+stopped 2020-04-10T07:06:56.825 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:06:56.825 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:06:56.825 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ all_admin_daemons cluster1 rbd mirror start 2020-04-10T07:06:56.825 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:56.825 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:06:56.826 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:06:56.828 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:0 rbd mirror start 2020-04-10T07:06:56.828 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:06:56.828 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:06:56.828 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:56.829 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:06:56.829 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:06:56.829 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:06:56.829 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:56.829 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:56.829 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:06:56.829 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:06:56.830 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:06:56.830 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:56.830 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:06:56.830 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:06:56.830 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster1 2020-04-10T07:06:56.830 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:06:56.830 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:56.831 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:56.831 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:06:56.831 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:06:56.831 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:06:56.831 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:56.831 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:56.831 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:06:56.831 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:06:56.832 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:06:56.832 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:06:56.832 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:06:56.832 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:06:56.832 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:06:56.845 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:06:56.845 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:06:56.845 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:06:56.845 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror start 2020-04-10T07:06:56.956 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:1 rbd mirror start 2020-04-10T07:06:56.957 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:1 2020-04-10T07:06:56.957 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:06:56.957 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:56.957 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 cluster instance 2020-04-10T07:06:56.957 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:06:56.957 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:06:56.958 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:56.958 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:56.958 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:06:56.958 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:06:56.958 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:06:56.958 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:56.958 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:06:56.958 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:06:56.959 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:1 cluster1 2020-04-10T07:06:56.959 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:1 2020-04-10T07:06:56.959 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:56.959 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:56.959 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2020-04-10T07:06:56.959 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:06:56.959 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:06:56.960 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:56.960 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:56.960 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:06:56.960 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:06:56.960 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:06:56.960 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:06:56.960 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:06:56.961 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:06:56.961 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2020-04-10T07:06:56.973 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:06:56.973 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:06:56.973 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:06:56.973 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror start 2020-04-10T07:06:57.066 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:2 rbd mirror start 2020-04-10T07:06:57.066 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:2 2020-04-10T07:06:57.066 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:06:57.067 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:57.067 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 cluster instance 2020-04-10T07:06:57.067 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:06:57.067 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:06:57.067 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:57.067 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:57.067 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:06:57.068 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:06:57.068 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:06:57.068 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:57.068 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:06:57.068 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:06:57.068 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:2 cluster1 2020-04-10T07:06:57.068 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:2 2020-04-10T07:06:57.068 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:57.069 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:57.069 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2020-04-10T07:06:57.069 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:06:57.069 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:06:57.069 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:57.069 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:57.069 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:06:57.070 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:06:57.070 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:06:57.070 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:06:57.070 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:06:57.070 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:06:57.070 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2020-04-10T07:06:57.087 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:06:57.087 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:06:57.088 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:06:57.088 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror start 2020-04-10T07:06:57.187 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:3 rbd mirror start 2020-04-10T07:06:57.187 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:3 2020-04-10T07:06:57.187 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:06:57.188 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:57.188 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:3 cluster instance 2020-04-10T07:06:57.188 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:3 2020-04-10T07:06:57.188 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:06:57.188 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:57.188 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:57.188 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:06:57.189 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 3 = cluster1:3 ] 2020-04-10T07:06:57.189 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:06:57.189 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:57.189 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=3 2020-04-10T07:06:57.189 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:06:57.189 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:3 cluster1 2020-04-10T07:06:57.189 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:3 2020-04-10T07:06:57.190 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:57.190 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:57.190 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:3 local_cluster instance 2020-04-10T07:06:57.190 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:3 2020-04-10T07:06:57.190 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:06:57.190 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:57.190 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:57.191 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:06:57.191 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 3 = cluster1:3 ] 2020-04-10T07:06:57.191 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:06:57.192 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:06:57.193 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=3 2020-04-10T07:06:57.193 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:06:57.193 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.3 admin socket 2020-04-10T07:06:57.207 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:06:57.208 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:06:57.208 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:06:57.208 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror start 2020-04-10T07:06:57.302 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_started cluster1 mirror test 2020-04-10T07:06:57.303 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:57.303 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:57.303 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:06:57.303 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2020-04-10T07:06:57.303 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:57.303 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:57.304 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:06:57.304 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state=started 2020-04-10T07:06:57.304 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local s 2020-04-10T07:06:57.304 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:06:58.304 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_image_replay_state cluster1 mirror test started 2020-04-10T07:06:58.304 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:58.304 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:58.304 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:06:58.305 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local test_state=started 2020-04-10T07:06:58.305 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_result 2020-04-10T07:06:58.305 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_state=stopped 2020-04-10T07:06:58.306 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2020-04-10T07:06:58.306 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_instance=cluster1 2020-04-10T07:06:58.306 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:06:58.307 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ + local cluster=cluster1 2020-04-10T07:06:58.307 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance=cluster1 2020-04-10T07:06:58.307 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:grep+ -ilocal state loop_instance 2020-04-10T07:06:58.307 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: 2020-04-10T07:06:58.307 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0 2020-04-10T07:06:58.307 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ cluster1 != cluster1 ] 2020-04-10T07:06:58.308 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:06:58.310 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2020-04-10T07:06:58.310 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:06:58.310 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:06:58.310 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:58.310 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:06:58.310 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:06:58.311 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:06:58.311 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:58.311 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:58.311 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:06:58.311 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:06:58.311 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:06:58.311 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:58.312 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:06:58.312 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:06:58.312 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster1 2020-04-10T07:06:58.312 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:06:58.312 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:58.312 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:58.313 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:06:58.313 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:06:58.313 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:06:58.313 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:58.313 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:58.313 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:06:58.313 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:06:58.314 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:06:58.314 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:06:58.314 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:06:58.314 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:06:58.314 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:06:58.327 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:06:58.328 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:06:58.328 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:06:58.328 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2020-04-10T07:06:58.424 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:no valid command found; 1 closest matches: 2020-04-10T07:06:58.425 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status mirror/test1 2020-04-10T07:06:58.425 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:06:58.428 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2020-04-10T07:06:58.428 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:1 2020-04-10T07:06:58.428 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:06:58.428 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:58.428 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 cluster instance 2020-04-10T07:06:58.428 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:06:58.429 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:06:58.429 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:58.429 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:58.429 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:06:58.429 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:06:58.429 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:06:58.429 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:58.430 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:06:58.430 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:06:58.430 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:1 cluster1 2020-04-10T07:06:58.430 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:1 2020-04-10T07:06:58.430 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:58.430 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:58.430 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2020-04-10T07:06:58.431 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:06:58.431 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:06:58.431 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:58.431 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:58.431 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:06:58.431 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:06:58.431 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:06:58.431 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:06:58.432 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:06:58.432 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:06:58.432 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2020-04-10T07:06:58.460 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:06:58.460 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:06:58.461 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:06:58.461 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2020-04-10T07:06:58.543 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:06:58.544 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ status_result= "state": "Replaying" 2020-04-10T07:06:58.545 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo "state": "Replaying" 2020-04-10T07:06:58.545 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep -i Replaying 2020-04-10T07:06:58.546 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "state": "Replaying" 2020-04-10T07:06:58.546 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ current_state=started 2020-04-10T07:06:58.547 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test started = started 2020-04-10T07:06:58.547 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:06:58.547 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_started cluster1 mirror test1 2020-04-10T07:06:58.547 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:58.547 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:58.547 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:06:58.547 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_state cluster1 mirror test1 started 2020-04-10T07:06:58.548 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:58.548 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:58.548 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:06:58.548 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state=started 2020-04-10T07:06:58.548 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local s 2020-04-10T07:06:58.548 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:06:59.547 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_image_replay_state cluster1 mirror test1 started 2020-04-10T07:06:59.548 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:59.548 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:59.548 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:06:59.548 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local test_state=started 2020-04-10T07:06:59.548 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_result 2020-04-10T07:06:59.548 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_state=stopped 2020-04-10T07:06:59.549 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test1 2020-04-10T07:06:59.549 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ + local cluster_instance=cluster1 2020-04-10T07:06:59.549 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:grep -i+ shift state 2020-04-10T07:06:59.549 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: 2020-04-10T07:06:59.549 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:59.549 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance=cluster1 2020-04-10T07:06:59.550 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local loop_instance 2020-04-10T07:06:59.550 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0 2020-04-10T07:06:59.552 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ cluster1 != cluster1 ] 2020-04-10T07:06:59.552 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:06:59.552 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test1 2020-04-10T07:06:59.552 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:06:59.553 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:06:59.553 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:59.553 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:06:59.553 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:06:59.553 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:06:59.553 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:59.553 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:59.554 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:06:59.554 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:06:59.554 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:06:59.554 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:59.554 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:06:59.554 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:06:59.555 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster1 2020-04-10T07:06:59.555 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:06:59.555 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:59.555 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:06:59.555 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:06:59.555 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:06:59.555 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:06:59.555 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:06:59.556 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:06:59.556 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:06:59.557 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:06:59.557 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:06:59.557 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:06:59.557 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:06:59.557 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:06:59.557 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:06:59.572 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:06:59.572 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:06:59.572 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:06:59.572 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test1 2020-04-10T07:06:59.677 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:06:59.678 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ status_result= "state": "Replaying" 2020-04-10T07:06:59.678 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo "state": "Replaying" 2020-04-10T07:06:59.678 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep -i Replaying 2020-04-10T07:06:59.687 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "state": "Replaying" 2020-04-10T07:06:59.687 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ current_state=started 2020-04-10T07:06:59.687 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test started = started 2020-04-10T07:06:59.687 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:06:59.688 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2020-04-10T07:06:59.688 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:06:59.688 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:06:59.688 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:06:59.688 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:06:59.688 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:06:59.689 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:06:59.689 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:07:00.682 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying 2020-04-10T07:07:00.682 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:00.682 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:00.682 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:00.682 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:07:00.683 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:07:00.683 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:07:00.683 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror-test.mirror_status 2020-04-10T07:07:00.683 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror-test.mirror_status 2020-04-10T07:07:00.683 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:07:00.684 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:07:00.685 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test 2020-04-10T07:07:00.685 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ tee /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:07:00.760 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:test: 2020-04-10T07:07:00.760 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: global_id: 687e9f7d-ae46-4324-b8f7-7c16605e0fa6 2020-04-10T07:07:00.760 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+replaying 2020-04-10T07:07:00.760 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: replaying, {"replay_state": "idle", "remote_snapshot_timestamp": 1586502409, "local_snapshot_timestamp": 1586502409} 2020-04-10T07:07:00.760 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: service: mirror.1 on smithi204 2020-04-10T07:07:00.761 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:06:57 2020-04-10T07:07:00.769 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ state: .*up+replaying /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:07:00.771 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+replaying 2020-04-10T07:07:00.771 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ description: .* /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:07:00.773 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: replaying, {"replay_state": "idle", "remote_snapshot_timestamp": 1586502409, "local_snapshot_timestamp": 1586502409} 2020-04-10T07:07:00.773 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:07:00.773 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+replaying 2020-04-10T07:07:00.773 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^up+ 2020-04-10T07:07:00.774 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+replaying 2020-04-10T07:07:00.775 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep service: *mirror..* on /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:07:00.776 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: service: mirror.1 on smithi204 2020-04-10T07:07:00.777 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:07:00.779 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update=2020-04-10 07:06:57 2020-04-10T07:07:00.779 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying 2020-04-10 07:06:57 2020-04-10T07:07:00.779 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:00.779 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:00.780 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:00.780 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:07:00.780 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local prev_last_update=2020-04-10 07:06:57 2020-04-10T07:07:00.780 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror.mirror_status 2020-04-10T07:07:00.781 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror.mirror_status 2020-04-10T07:07:00.781 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:07:00.782 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror.mirror_status 2020-04-10T07:07:00.782 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2020-04-10T07:07:00.825 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: failed to query services: (13) Permission denied 2020-04-10T07:07:00.874 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update state 2020-04-10T07:07:00.874 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2020-04-10T07:07:00.879 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ last_update=2020-04-10 07:06:57 2020-04-10T07:07:00.880 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2020-04-10T07:07:00.885 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ state=up+replaying 2020-04-10T07:07:00.885 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo+ up+replaying 2020-04-10T07:07:00.886 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:grep up+replaying 2020-04-10T07:07:00.887 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+replaying 2020-04-10T07:07:00.887 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:00.887 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:00.887 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test1 up+replaying 2020-04-10T07:07:00.887 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:00.887 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:00.888 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:07:00.888 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:07:00.888 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:07:00.888 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:07:00.888 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:07:01.889 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_status_in_pool_dir cluster1 mirror test1 up+replaying 2020-04-10T07:07:01.889 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:01.889 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:01.889 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:07:01.889 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:07:01.889 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:07:01.889 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:07:01.890 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror-test1.mirror_status 2020-04-10T07:07:01.890 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror-test1.mirror_status 2020-04-10T07:07:01.890 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:07:01.891 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.mirror_status 2020-04-10T07:07:01.891 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test1 2020-04-10T07:07:01.891 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ tee /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.mirror_status 2020-04-10T07:07:01.960 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:test1: 2020-04-10T07:07:01.960 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: global_id: 56419286-13e9-49f6-a20d-2abe6ede083e 2020-04-10T07:07:01.961 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+replaying 2020-04-10T07:07:01.961 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: replaying, {"replay_state": "idle", "remote_snapshot_timestamp": 1586502402, "local_snapshot_timestamp": 1586502402} 2020-04-10T07:07:01.961 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: service: mirror.0 on smithi204 2020-04-10T07:07:01.961 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:06:57 2020-04-10T07:07:01.969 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ state: .*up+replaying /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.mirror_status 2020-04-10T07:07:01.971 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+replaying 2020-04-10T07:07:01.971 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ description: .* /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.mirror_status 2020-04-10T07:07:01.973 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: replaying, {"replay_state": "idle", "remote_snapshot_timestamp": 1586502402, "local_snapshot_timestamp": 1586502402} 2020-04-10T07:07:01.973 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:07:01.974 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+replaying 2020-04-10T07:07:01.974 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^up+ 2020-04-10T07:07:01.975 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+replaying 2020-04-10T07:07:01.975 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep service: *mirror..* on /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.mirror_status 2020-04-10T07:07:01.976 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: service: mirror.0 on smithi204 2020-04-10T07:07:01.977 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.mirror_status 2020-04-10T07:07:01.979 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update=2020-04-10 07:06:57 2020-04-10T07:07:01.980 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test1 up+replaying 2020-04-10 07:06:57 2020-04-10T07:07:01.980 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:01.980 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:01.980 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:07:01.980 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:07:01.980 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local prev_last_update=2020-04-10 07:06:57 2020-04-10T07:07:01.981 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror.mirror_status 2020-04-10T07:07:01.981 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror.mirror_status 2020-04-10T07:07:01.981 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:07:01.982 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror.mirror_status 2020-04-10T07:07:01.982 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2020-04-10T07:07:02.025 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: failed to query services: (13) Permission denied 2020-04-10T07:07:02.074 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update state 2020-04-10T07:07:02.074 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test1']/last_update 2020-04-10T07:07:02.080 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ last_update=2020-04-10 07:06:57 2020-04-10T07:07:02.080 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test1']/state 2020-04-10T07:07:02.084 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ state=up+replaying 2020-04-10T07:07:02.084 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+replaying 2020-04-10T07:07:02.084 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep up+replaying 2020-04-10T07:07:02.086 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+replaying 2020-04-10T07:07:02.086 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:02.086 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:02.086 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ all_admin_daemons cluster1 rbd mirror restart 2020-04-10T07:07:02.086 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:02.087 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:02.087 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:07:02.089 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:0 rbd mirror restart 2020-04-10T07:07:02.089 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:07:02.089 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:02.089 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:02.089 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:07:02.090 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:02.090 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:02.090 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:02.090 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:02.090 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:02.090 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:02.090 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:02.091 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:02.091 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:02.091 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:02.091 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster1 2020-04-10T07:07:02.091 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:07:02.091 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:02.092 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:02.092 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:07:02.092 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:02.092 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:02.092 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:02.092 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:02.093 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:02.093 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:02.093 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:02.093 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:02.093 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:02.093 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:02.093 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:07:02.108 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:02.109 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:02.109 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:02.109 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror restart 2020-04-10T07:07:02.224 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:1 rbd mirror restart 2020-04-10T07:07:02.224 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:1 2020-04-10T07:07:02.225 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:02.225 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:02.225 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 cluster instance 2020-04-10T07:07:02.225 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:07:02.225 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:02.225 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:02.225 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:02.226 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:02.226 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:07:02.226 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:02.226 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:02.226 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:07:02.226 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:02.226 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:1 cluster1 2020-04-10T07:07:02.227 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:1 2020-04-10T07:07:02.227 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:02.227 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:02.227 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2020-04-10T07:07:02.227 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:07:02.227 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:02.227 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:02.227 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:02.228 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:02.228 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:07:02.228 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:02.228 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:02.228 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:07:02.228 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:02.228 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2020-04-10T07:07:02.241 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:02.241 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:02.241 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:02.241 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror restart 2020-04-10T07:07:02.335 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:2 rbd mirror restart 2020-04-10T07:07:02.335 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:2 2020-04-10T07:07:02.335 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:02.335 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:02.335 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 cluster instance 2020-04-10T07:07:02.336 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:07:02.336 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:02.336 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:02.336 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:02.336 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:07:02.336 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:07:02.336 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:02.337 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:02.337 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:07:02.337 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:07:02.337 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:2 cluster1 2020-04-10T07:07:02.337 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:2 2020-04-10T07:07:02.337 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:02.337 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:02.338 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2020-04-10T07:07:02.338 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:07:02.338 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:02.338 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:02.338 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:02.338 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:07:02.338 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:07:02.339 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:02.339 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:02.339 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:07:02.339 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:07:02.339 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2020-04-10T07:07:02.348 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:07:02.349 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:07:02.349 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:07:02.349 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror restart 2020-04-10T07:07:02.441 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:3 rbd mirror restart 2020-04-10T07:07:02.441 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:3 2020-04-10T07:07:02.441 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:02.442 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:02.442 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:3 cluster instance 2020-04-10T07:07:02.442 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:3 2020-04-10T07:07:02.442 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:02.442 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:02.442 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:02.442 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:07:02.443 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 3 = cluster1:3 ] 2020-04-10T07:07:02.443 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:02.443 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:02.443 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=3 2020-04-10T07:07:02.443 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:07:02.443 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:3 cluster1 2020-04-10T07:07:02.443 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:3 2020-04-10T07:07:02.443 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:02.444 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:02.444 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:3 local_cluster instance 2020-04-10T07:07:02.444 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:3 2020-04-10T07:07:02.444 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:02.444 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:02.444 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:02.444 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:07:02.445 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 3 = cluster1:3 ] 2020-04-10T07:07:02.445 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:02.445 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:02.445 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=3 2020-04-10T07:07:02.445 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:07:02.445 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.3 admin socket 2020-04-10T07:07:02.457 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:07:02.458 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:07:02.458 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:07:02.458 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror restart 2020-04-10T07:07:02.552 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_started cluster1 mirror test 2020-04-10T07:07:02.552 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:02.552 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:02.552 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:02.552 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2020-04-10T07:07:02.553 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:02.553 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:02.553 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:02.553 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state=started 2020-04-10T07:07:02.553 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local s 2020-04-10T07:07:02.553 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:07:03.553 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_image_replay_state cluster1 mirror test started 2020-04-10T07:07:03.553 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:03.553 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:03.553 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:03.553 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local test_state=started 2020-04-10T07:07:03.554 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_result 2020-04-10T07:07:03.554 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_state=stopped 2020-04-10T07:07:03.555 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2020-04-10T07:07:03.555 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_instance=cluster1 2020-04-10T07:07:03.556 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ + shift 2020-04-10T07:07:03.556 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:03.556 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:grep+ -ilocal state instance=cluster1 2020-04-10T07:07:03.556 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: 2020-04-10T07:07:03.556 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local loop_instance 2020-04-10T07:07:03.556 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0 2020-04-10T07:07:03.556 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ cluster1 != cluster1 ] 2020-04-10T07:07:03.557 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:07:03.559 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2020-04-10T07:07:03.559 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:07:03.559 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:03.560 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:03.560 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:07:03.560 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:03.560 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:03.560 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:03.560 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:03.560 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:03.561 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:03.561 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:03.561 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:03.561 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:03.561 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:03.561 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster1 2020-04-10T07:07:03.561 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:07:03.562 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:03.562 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:03.562 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:07:03.562 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:03.562 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:03.562 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:03.562 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:03.563 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:03.563 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:03.563 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:03.563 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:03.563 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:03.563 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:03.563 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:07:03.578 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:03.578 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:03.579 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:03.579 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2020-04-10T07:07:03.674 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:no valid command found; 1 closest matches: 2020-04-10T07:07:03.674 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status mirror/test1 2020-04-10T07:07:03.674 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:07:03.677 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2020-04-10T07:07:03.678 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:1 2020-04-10T07:07:03.678 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:03.678 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:03.678 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 cluster instance 2020-04-10T07:07:03.678 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:07:03.678 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:03.679 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:03.679 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:03.679 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:03.679 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:07:03.679 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:03.679 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:03.679 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:07:03.679 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:03.680 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:1 cluster1 2020-04-10T07:07:03.680 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:1 2020-04-10T07:07:03.680 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:03.680 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:03.680 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2020-04-10T07:07:03.680 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:07:03.680 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:03.681 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:03.681 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:03.681 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:03.681 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:07:03.681 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:03.681 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:03.681 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:07:03.682 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:03.682 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2020-04-10T07:07:03.699 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:03.699 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:03.699 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:03.699 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2020-04-10T07:07:03.800 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:03.800 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ status_result= "state": "Replaying" 2020-04-10T07:07:03.800 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo "state": "Replaying" 2020-04-10T07:07:03.800 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep -i Replaying 2020-04-10T07:07:03.801 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "state": "Replaying" 2020-04-10T07:07:03.802 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ current_state=started 2020-04-10T07:07:03.802 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test started = started 2020-04-10T07:07:03.802 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:03.802 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_started cluster1 mirror test1 2020-04-10T07:07:03.802 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:03.802 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:03.802 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:07:03.803 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_state cluster1 mirror test1 started 2020-04-10T07:07:03.803 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:03.803 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:03.803 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:07:03.803 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state=started 2020-04-10T07:07:03.803 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local s 2020-04-10T07:07:03.803 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:07:04.803 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_image_replay_state cluster1 mirror test1 started 2020-04-10T07:07:04.803 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:04.804 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:04.804 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:07:04.804 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local test_state=started 2020-04-10T07:07:04.804 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_result 2020-04-10T07:07:04.804 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_state=stopped 2020-04-10T07:07:04.804 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test1 2020-04-10T07:07:04.805 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_instance=cluster1 2020-04-10T07:07:04.805 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:04.805 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ + local cluster=cluster1 2020-04-10T07:07:04.805 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance=cluster1grep 2020-04-10T07:07:04.805 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: -i+ statelocal 2020-04-10T07:07:04.805 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: loop_instance 2020-04-10T07:07:04.805 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0 2020-04-10T07:07:04.807 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ cluster1 != cluster1 ] 2020-04-10T07:07:04.807 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:07:04.807 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test1 2020-04-10T07:07:04.807 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:07:04.808 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:04.808 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:04.808 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:07:04.808 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:04.808 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:04.808 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:04.809 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:04.809 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:04.809 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:04.809 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:04.809 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:04.810 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:04.810 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:04.810 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster1 2020-04-10T07:07:04.810 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:07:04.810 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:04.810 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:04.810 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:07:04.811 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:04.812 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:04.812 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:04.812 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:04.812 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:04.812 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:04.812 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:04.813 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:04.813 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:04.813 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:04.813 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:07:04.829 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:04.829 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:04.830 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:04.830 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test1 2020-04-10T07:07:04.933 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:04.934 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ status_result= "state": "Replaying" 2020-04-10T07:07:04.934 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ + echo "state": "Replaying" 2020-04-10T07:07:04.934 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:grep -i Replaying 2020-04-10T07:07:04.936 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "state": "Replaying" 2020-04-10T07:07:04.936 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ current_state=started 2020-04-10T07:07:04.936 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test started = started 2020-04-10T07:07:04.936 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:04.937 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2020-04-10T07:07:04.937 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:04.937 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:04.937 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:04.937 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:07:04.937 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:07:04.937 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:07:04.938 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:07:05.936 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying 2020-04-10T07:07:05.937 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:05.937 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:05.937 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:05.937 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:07:05.937 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:07:05.938 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:07:05.938 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror-test.mirror_status 2020-04-10T07:07:05.938 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror-test.mirror_status 2020-04-10T07:07:05.938 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:07:05.939 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:07:05.939 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test 2020-04-10T07:07:05.939 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ tee /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:07:06.014 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:test: 2020-04-10T07:07:06.014 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: global_id: 687e9f7d-ae46-4324-b8f7-7c16605e0fa6 2020-04-10T07:07:06.014 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+replaying 2020-04-10T07:07:06.014 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: replaying, {"replay_state": "idle", "remote_snapshot_timestamp": 1586502409, "local_snapshot_timestamp": 1586502409} 2020-04-10T07:07:06.015 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: service: mirror.1 on smithi204 2020-04-10T07:07:06.015 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:07:02 2020-04-10T07:07:06.022 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ state: .*up+replaying /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:07:06.024 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+replaying 2020-04-10T07:07:06.025 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ description: .* /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:07:06.026 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: replaying, {"replay_state": "idle", "remote_snapshot_timestamp": 1586502409, "local_snapshot_timestamp": 1586502409} 2020-04-10T07:07:06.026 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:07:06.027 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+replaying 2020-04-10T07:07:06.027 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^up+ 2020-04-10T07:07:06.028 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+replaying 2020-04-10T07:07:06.028 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep service: *mirror..* on /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:07:06.030 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: service: mirror.1 on smithi204 2020-04-10T07:07:06.030 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:07:06.032 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update=2020-04-10 07:07:02 2020-04-10T07:07:06.032 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying 2020-04-10 07:07:02 2020-04-10T07:07:06.032 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:06.032 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:06.032 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:06.033 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:07:06.033 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local prev_last_update=2020-04-10 07:07:02 2020-04-10T07:07:06.033 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror.mirror_status 2020-04-10T07:07:06.033 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror.mirror_status 2020-04-10T07:07:06.034 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:07:06.034 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror.mirror_status 2020-04-10T07:07:06.035 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2020-04-10T07:07:06.075 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: failed to query services: (13) Permission denied 2020-04-10T07:07:06.120 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update state 2020-04-10T07:07:06.120 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2020-04-10T07:07:06.125 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ last_update=2020-04-10 07:07:02 2020-04-10T07:07:06.125 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2020-04-10T07:07:06.131 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ state=up+replaying 2020-04-10T07:07:06.131 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+replaying 2020-04-10T07:07:06.131 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep up+replaying 2020-04-10T07:07:06.133 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+replaying 2020-04-10T07:07:06.133 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:06.133 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:06.133 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test1 up+replaying 2020-04-10T07:07:06.133 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:06.133 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:06.134 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:07:06.134 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:07:06.134 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:07:06.134 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:07:06.134 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:07:07.138 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_status_in_pool_dir cluster1 mirror test1 up+replaying 2020-04-10T07:07:07.138 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:07.138 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:07.138 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:07:07.138 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:07:07.139 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:07:07.139 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:07:07.139 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror-test1.mirror_status 2020-04-10T07:07:07.139 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror-test1.mirror_status 2020-04-10T07:07:07.139 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:07:07.139 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.mirror_status 2020-04-10T07:07:07.139 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test1 2020-04-10T07:07:07.140 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ tee /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.mirror_status 2020-04-10T07:07:07.209 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:test1: 2020-04-10T07:07:07.209 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: global_id: 56419286-13e9-49f6-a20d-2abe6ede083e 2020-04-10T07:07:07.209 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+replaying 2020-04-10T07:07:07.210 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: replaying, {"replay_state": "idle", "remote_snapshot_timestamp": 1586502402, "local_snapshot_timestamp": 1586502402} 2020-04-10T07:07:07.210 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: service: mirror.0 on smithi204 2020-04-10T07:07:07.210 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:07:02 2020-04-10T07:07:07.217 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ state: .*up+replaying /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.mirror_status 2020-04-10T07:07:07.218 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+replaying 2020-04-10T07:07:07.218 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ description: .* /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.mirror_status 2020-04-10T07:07:07.220 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: replaying, {"replay_state": "idle", "remote_snapshot_timestamp": 1586502402, "local_snapshot_timestamp": 1586502402} 2020-04-10T07:07:07.221 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:07:07.221 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+replaying 2020-04-10T07:07:07.221 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^up+ 2020-04-10T07:07:07.222 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+replaying 2020-04-10T07:07:07.222 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep service: *mirror..* on /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.mirror_status 2020-04-10T07:07:07.224 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: service: mirror.0 on smithi204 2020-04-10T07:07:07.224 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.mirror_status 2020-04-10T07:07:07.225 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update=2020-04-10 07:07:02 2020-04-10T07:07:07.226 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test1 up+replaying 2020-04-10 07:07:02 2020-04-10T07:07:07.226 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:07.226 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:07.226 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:07:07.226 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:07:07.226 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local prev_last_update=2020-04-10 07:07:02 2020-04-10T07:07:07.227 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror.mirror_status 2020-04-10T07:07:07.227 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror.mirror_status 2020-04-10T07:07:07.227 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:07:07.227 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror.mirror_status 2020-04-10T07:07:07.228 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2020-04-10T07:07:07.270 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: failed to query services: (13) Permission denied 2020-04-10T07:07:07.321 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update state 2020-04-10T07:07:07.322 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test1']/last_update 2020-04-10T07:07:07.327 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ last_update=2020-04-10 07:07:02 2020-04-10T07:07:07.328 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test1']/state 2020-04-10T07:07:07.331 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ state=up+replaying 2020-04-10T07:07:07.331 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ + echo up+replaying 2020-04-10T07:07:07.331 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:grep up+replaying 2020-04-10T07:07:07.333 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+replaying 2020-04-10T07:07:07.333 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:07.333 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:07.334 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ all_admin_daemons cluster1 rbd mirror stop 2020-04-10T07:07:07.334 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:07.334 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:07.334 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:07:07.336 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:0 rbd mirror stop 2020-04-10T07:07:07.336 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:07:07.336 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:07.336 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:07.337 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:07:07.337 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:07.337 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:07.337 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:07.337 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:07.337 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:07.337 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:07.338 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:07.338 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:07.339 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:07.339 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:07.339 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster1 2020-04-10T07:07:07.340 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:07:07.340 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:07.340 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:07.340 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:07:07.340 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:07.340 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:07.340 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:07.341 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:07.341 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:07.341 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:07.341 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:07.341 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:07.341 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:07.341 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:07.342 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:07:07.354 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:07.354 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:07.354 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:07.354 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror stop 2020-04-10T07:07:07.459 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:1 rbd mirror stop 2020-04-10T07:07:07.459 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:1 2020-04-10T07:07:07.459 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:07.460 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:07.460 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 cluster instance 2020-04-10T07:07:07.460 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:07:07.460 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:07.460 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:07.460 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:07.461 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:07.461 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:07:07.461 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:07.461 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:07.461 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:07:07.461 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:07.461 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:1 cluster1 2020-04-10T07:07:07.462 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:1 2020-04-10T07:07:07.462 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:07.462 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:07.462 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2020-04-10T07:07:07.462 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:07:07.462 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:07.462 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:07.463 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:07.463 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:07.463 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:07:07.463 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:07.463 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:07.463 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:07:07.463 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:07.464 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2020-04-10T07:07:07.477 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:07.477 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:07.477 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:07.478 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror stop 2020-04-10T07:07:07.578 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:2 rbd mirror stop 2020-04-10T07:07:07.578 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:2 2020-04-10T07:07:07.578 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:07.578 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:07.578 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 cluster instance 2020-04-10T07:07:07.578 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:07:07.579 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:07.579 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:07.579 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:07.579 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:07:07.579 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:07:07.579 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:07.580 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:07.580 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:07:07.580 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:07:07.580 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:2 cluster1 2020-04-10T07:07:07.580 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:2 2020-04-10T07:07:07.581 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:07.581 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:07.581 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2020-04-10T07:07:07.581 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:07:07.581 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:07.581 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:07.581 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:07.582 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:07:07.582 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:07:07.582 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:07.582 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:07.582 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:07:07.582 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:07:07.583 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2020-04-10T07:07:07.592 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:07:07.592 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:07:07.593 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:07:07.593 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror stop 2020-04-10T07:07:07.684 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:3 rbd mirror stop 2020-04-10T07:07:07.684 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:3 2020-04-10T07:07:07.685 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:07.685 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:07.685 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:3 cluster instance 2020-04-10T07:07:07.685 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:3 2020-04-10T07:07:07.685 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:07.685 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:07.685 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:07.686 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:07:07.686 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 3 = cluster1:3 ] 2020-04-10T07:07:07.686 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:07.686 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:07.686 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=3 2020-04-10T07:07:07.686 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:07:07.686 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:3 cluster1 2020-04-10T07:07:07.687 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:3 2020-04-10T07:07:07.687 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:07.687 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:07.687 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:3 local_cluster instance 2020-04-10T07:07:07.687 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:3 2020-04-10T07:07:07.687 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:07.687 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:07.688 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:07.688 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:07:07.688 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 3 = cluster1:3 ] 2020-04-10T07:07:07.688 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:07.688 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:07.688 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=3 2020-04-10T07:07:07.688 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:07:07.689 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.3 admin socket 2020-04-10T07:07:07.706 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:07:07.706 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:07:07.707 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:07:07.707 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror stop 2020-04-10T07:07:07.803 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_stopped cluster1 mirror test 2020-04-10T07:07:07.803 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:07.804 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:07.804 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:07.804 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_state cluster1 mirror test stopped 2020-04-10T07:07:07.804 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:07.804 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:07.804 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:07.804 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state=stopped 2020-04-10T07:07:07.805 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local s 2020-04-10T07:07:07.805 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:07:08.805 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_image_replay_state cluster1 mirror test stopped 2020-04-10T07:07:08.805 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:08.805 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:08.805 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:08.805 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local test_state=stopped 2020-04-10T07:07:08.805 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_result 2020-04-10T07:07:08.806 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_state=stopped 2020-04-10T07:07:08.807 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2020-04-10T07:07:08.807 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_instance=cluster1 2020-04-10T07:07:08.807 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ + shift 2020-04-10T07:07:08.807 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:grep -i state 2020-04-10T07:07:08.807 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:08.807 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance=cluster1 2020-04-10T07:07:08.808 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local loop_instance 2020-04-10T07:07:08.808 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0 2020-04-10T07:07:08.808 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ cluster1 != cluster1 ] 2020-04-10T07:07:08.808 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:07:08.810 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2020-04-10T07:07:08.810 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:07:08.810 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:08.810 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:08.810 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:07:08.810 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:08.811 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:08.811 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:08.811 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:08.811 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:08.811 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:08.811 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:08.811 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:08.812 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:08.812 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:08.812 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster1 2020-04-10T07:07:08.812 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:07:08.812 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:08.812 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:08.812 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:07:08.813 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:08.813 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:08.813 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:08.814 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:08.814 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:08.814 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:08.814 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:08.815 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:08.815 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:08.815 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:08.815 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:07:08.833 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:08.833 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:08.833 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:08.833 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2020-04-10T07:07:08.926 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:no valid command found; 1 closest matches: 2020-04-10T07:07:08.927 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status mirror/test1 2020-04-10T07:07:08.927 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:07:08.929 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2020-04-10T07:07:08.929 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:1 2020-04-10T07:07:08.929 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:08.929 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:08.930 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 cluster instance 2020-04-10T07:07:08.930 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:07:08.930 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:08.930 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:08.930 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:08.930 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:08.930 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:07:08.931 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:08.931 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:08.931 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:07:08.931 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:08.931 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:1 cluster1 2020-04-10T07:07:08.931 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:1 2020-04-10T07:07:08.931 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:08.931 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:08.932 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2020-04-10T07:07:08.932 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:07:08.932 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:08.932 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:08.932 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:08.932 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:08.932 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:07:08.933 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:08.933 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:08.933 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:07:08.933 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:08.933 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2020-04-10T07:07:08.951 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:08.951 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:08.951 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:08.951 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2020-04-10T07:07:09.064 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:09.064 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ status_result= "state": "Stopped" 2020-04-10T07:07:09.065 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo "state": "Stopped" 2020-04-10T07:07:09.065 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep -i Replaying 2020-04-10T07:07:09.066 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test stopped = stopped 2020-04-10T07:07:09.066 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:09.066 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_stopped cluster1 mirror test1 2020-04-10T07:07:09.066 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:09.066 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:09.066 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:07:09.067 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_state cluster1 mirror test1 stopped 2020-04-10T07:07:09.067 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:09.067 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:09.067 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:07:09.067 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state=stopped 2020-04-10T07:07:09.067 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local s 2020-04-10T07:07:09.067 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:07:10.067 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_image_replay_state cluster1 mirror test1 stopped 2020-04-10T07:07:10.067 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:10.067 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:10.067 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:07:10.067 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local test_state=stopped 2020-04-10T07:07:10.068 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_result 2020-04-10T07:07:10.068 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_state=stopped 2020-04-10T07:07:10.069 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test1 2020-04-10T07:07:10.069 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_instance=cluster1 2020-04-10T07:07:10.069 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:10.069 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ + local cluster=cluster1 2020-04-10T07:07:10.069 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance=cluster1 2020-04-10T07:07:10.070 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local loop_instance 2020-04-10T07:07:10.070 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:grep+ sleep -i 0 state 2020-04-10T07:07:10.070 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: 2020-04-10T07:07:10.070 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ cluster1 != cluster1 ] 2020-04-10T07:07:10.070 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:07:10.072 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test1 2020-04-10T07:07:10.072 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:07:10.072 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:10.072 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:10.073 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:07:10.073 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:10.073 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:10.073 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:10.073 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:10.073 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:10.074 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:10.074 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:10.074 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:10.074 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:10.074 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:10.074 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster1 2020-04-10T07:07:10.074 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:07:10.074 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:10.075 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:10.075 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:07:10.075 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:10.075 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:10.075 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:10.075 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:10.075 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:10.076 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:10.076 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:10.076 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:10.076 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:10.076 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:10.076 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:07:10.090 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:10.091 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:10.091 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:10.091 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test1 2020-04-10T07:07:10.196 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:10.197 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ status_result= "state": "Stopped" 2020-04-10T07:07:10.197 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo "state": "Stopped" 2020-04-10T07:07:10.197 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep -i Replaying 2020-04-10T07:07:10.200 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test stopped = stopped 2020-04-10T07:07:10.200 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:10.200 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+stopped 2020-04-10T07:07:10.200 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:10.200 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:10.201 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:10.201 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+stopped 2020-04-10T07:07:10.201 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:07:10.201 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:07:10.201 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:07:11.200 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_status_in_pool_dir cluster1 mirror test up+stopped 2020-04-10T07:07:11.200 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:11.201 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:11.201 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:11.201 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+stopped 2020-04-10T07:07:11.201 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:07:11.201 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:07:11.201 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror-test.mirror_status 2020-04-10T07:07:11.202 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror-test.mirror_status 2020-04-10T07:07:11.202 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:07:11.202 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:07:11.203 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test 2020-04-10T07:07:11.203 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ tee /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:07:11.275 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:test: 2020-04-10T07:07:11.275 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: global_id: 687e9f7d-ae46-4324-b8f7-7c16605e0fa6 2020-04-10T07:07:11.275 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+stopped 2020-04-10T07:07:11.275 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: stopped 2020-04-10T07:07:11.275 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: service: mirror.1 on smithi204 2020-04-10T07:07:11.276 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:07:07 2020-04-10T07:07:11.283 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ state: .*up+stopped /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:07:11.285 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+stopped 2020-04-10T07:07:11.286 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ description: .* /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:07:11.287 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: stopped 2020-04-10T07:07:11.287 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:07:11.288 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+stopped 2020-04-10T07:07:11.288 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^up+ 2020-04-10T07:07:11.289 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+stopped 2020-04-10T07:07:11.289 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep service: *mirror..* on /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:07:11.291 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: service: mirror.1 on smithi204 2020-04-10T07:07:11.291 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:07:11.294 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update=2020-04-10 07:07:07 2020-04-10T07:07:11.294 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+stopped 2020-04-10 07:07:07 2020-04-10T07:07:11.294 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:11.294 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:11.295 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:11.295 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+stopped 2020-04-10T07:07:11.295 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local prev_last_update=2020-04-10 07:07:07 2020-04-10T07:07:11.295 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror.mirror_status 2020-04-10T07:07:11.295 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror.mirror_status 2020-04-10T07:07:11.295 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:07:11.296 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror.mirror_status 2020-04-10T07:07:11.296 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2020-04-10T07:07:11.340 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: failed to query services: (13) Permission denied 2020-04-10T07:07:11.390 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update state 2020-04-10T07:07:11.390 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2020-04-10T07:07:11.396 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ last_update=2020-04-10 07:07:07 2020-04-10T07:07:11.396 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2020-04-10T07:07:11.401 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ state=up+stopped 2020-04-10T07:07:11.402 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+stopped 2020-04-10T07:07:11.402 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep up+stopped 2020-04-10T07:07:11.403 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+stopped 2020-04-10T07:07:11.404 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:11.404 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:11.404 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test1 up+stopped 2020-04-10T07:07:11.404 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:11.404 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:11.404 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:07:11.404 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+stopped 2020-04-10T07:07:11.405 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:07:11.405 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:07:11.405 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:07:12.405 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_status_in_pool_dir cluster1 mirror test1 up+stopped 2020-04-10T07:07:12.405 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:12.405 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:12.405 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:07:12.406 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+stopped 2020-04-10T07:07:12.406 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:07:12.406 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:07:12.406 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror-test1.mirror_status 2020-04-10T07:07:12.406 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror-test1.mirror_status 2020-04-10T07:07:12.406 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:07:12.407 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.mirror_status 2020-04-10T07:07:12.407 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test1 2020-04-10T07:07:12.408 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ tee /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.mirror_status 2020-04-10T07:07:12.482 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:test1: 2020-04-10T07:07:12.482 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: global_id: 56419286-13e9-49f6-a20d-2abe6ede083e 2020-04-10T07:07:12.482 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+stopped 2020-04-10T07:07:12.482 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: stopped 2020-04-10T07:07:12.482 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: service: mirror.0 on smithi204 2020-04-10T07:07:12.482 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:07:07 2020-04-10T07:07:12.489 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ state: .*up+stopped /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.mirror_status 2020-04-10T07:07:12.491 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+stopped 2020-04-10T07:07:12.491 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ description: .* /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.mirror_status 2020-04-10T07:07:12.493 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: stopped 2020-04-10T07:07:12.493 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:07:12.494 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+stopped 2020-04-10T07:07:12.494 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^up+ 2020-04-10T07:07:12.495 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+stopped 2020-04-10T07:07:12.495 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep service: *mirror..* on /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.mirror_status 2020-04-10T07:07:12.496 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: service: mirror.0 on smithi204 2020-04-10T07:07:12.497 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.mirror_status 2020-04-10T07:07:12.500 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update=2020-04-10 07:07:07 2020-04-10T07:07:12.500 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test1 up+stopped 2020-04-10 07:07:07 2020-04-10T07:07:12.500 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:12.500 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:12.500 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:07:12.500 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+stopped 2020-04-10T07:07:12.500 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local prev_last_update=2020-04-10 07:07:07 2020-04-10T07:07:12.501 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror.mirror_status 2020-04-10T07:07:12.501 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror.mirror_status 2020-04-10T07:07:12.502 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:07:12.502 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror.mirror_status 2020-04-10T07:07:12.502 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2020-04-10T07:07:12.543 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: failed to query services: (13) Permission denied 2020-04-10T07:07:12.594 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update state 2020-04-10T07:07:12.595 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test1']/last_update 2020-04-10T07:07:12.599 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ last_update=2020-04-10 07:07:07 2020-04-10T07:07:12.600 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test1']/state 2020-04-10T07:07:12.605 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ state=up+stopped 2020-04-10T07:07:12.606 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+stopped 2020-04-10T07:07:12.606 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep up+stopped 2020-04-10T07:07:12.607 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+stopped 2020-04-10T07:07:12.608 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:12.608 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:12.608 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ all_admin_daemons cluster1 rbd mirror restart 2020-04-10T07:07:12.608 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:12.608 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:12.608 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:07:12.610 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:0 rbd mirror restart 2020-04-10T07:07:12.610 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:07:12.610 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:12.611 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:12.611 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:07:12.611 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:12.611 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:12.611 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:12.611 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:12.611 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:12.612 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:12.612 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:12.612 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:12.612 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:12.612 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:12.612 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster1 2020-04-10T07:07:12.612 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:07:12.613 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:12.613 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:12.613 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:07:12.613 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:12.613 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:12.613 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:12.613 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:12.614 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:12.614 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:12.614 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:12.614 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:12.614 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:12.614 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:12.614 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:07:12.632 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:12.632 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:12.632 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:12.632 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror restart 2020-04-10T07:07:12.738 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:1 rbd mirror restart 2020-04-10T07:07:12.739 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:1 2020-04-10T07:07:12.739 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:12.739 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:12.739 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 cluster instance 2020-04-10T07:07:12.739 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:07:12.739 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:12.739 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:12.740 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:12.740 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:12.740 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:07:12.740 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:12.740 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:12.740 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:07:12.740 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:12.741 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:1 cluster1 2020-04-10T07:07:12.741 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:1 2020-04-10T07:07:12.741 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:12.741 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:12.741 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2020-04-10T07:07:12.741 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:07:12.741 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:12.742 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:12.742 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:12.742 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:12.742 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:07:12.742 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:12.742 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:12.742 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:07:12.743 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:12.743 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2020-04-10T07:07:12.753 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:12.753 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:12.753 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:12.754 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror restart 2020-04-10T07:07:12.858 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:2 rbd mirror restart 2020-04-10T07:07:12.858 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:2 2020-04-10T07:07:12.858 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:12.859 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:12.859 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 cluster instance 2020-04-10T07:07:12.859 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:07:12.859 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:12.859 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:12.859 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:12.859 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:07:12.860 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:07:12.860 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:12.860 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:12.860 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:07:12.860 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:07:12.860 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:2 cluster1 2020-04-10T07:07:12.860 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:2 2020-04-10T07:07:12.861 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:12.861 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:12.861 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2020-04-10T07:07:12.861 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:07:12.861 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:12.861 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:12.862 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:12.862 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:07:12.862 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:07:12.862 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:12.862 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:12.862 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:07:12.863 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:07:12.863 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2020-04-10T07:07:12.875 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:07:12.875 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:07:12.876 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:07:12.876 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror restart 2020-04-10T07:07:12.965 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:3 rbd mirror restart 2020-04-10T07:07:12.965 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:3 2020-04-10T07:07:12.965 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:12.965 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:12.966 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:3 cluster instance 2020-04-10T07:07:12.966 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:3 2020-04-10T07:07:12.966 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:12.966 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:12.966 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:12.966 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:07:12.966 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 3 = cluster1:3 ] 2020-04-10T07:07:12.967 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:12.967 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:12.967 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=3 2020-04-10T07:07:12.967 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:07:12.967 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:3 cluster1 2020-04-10T07:07:12.967 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:3 2020-04-10T07:07:12.967 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:12.967 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:12.968 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:3 local_cluster instance 2020-04-10T07:07:12.968 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:3 2020-04-10T07:07:12.968 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:12.968 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:12.968 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:12.968 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:07:12.969 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 3 = cluster1:3 ] 2020-04-10T07:07:12.969 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:12.969 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:12.969 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=3 2020-04-10T07:07:12.969 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:07:12.969 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.3 admin socket 2020-04-10T07:07:12.985 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:07:12.986 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:07:12.986 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:07:12.986 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror restart 2020-04-10T07:07:13.079 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_started cluster1 mirror test 2020-04-10T07:07:13.079 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:13.079 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:13.079 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:13.080 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2020-04-10T07:07:13.080 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:13.080 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:13.080 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:13.080 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state=started 2020-04-10T07:07:13.080 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local s 2020-04-10T07:07:13.081 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:07:14.081 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_image_replay_state cluster1 mirror test started 2020-04-10T07:07:14.081 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:14.081 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:14.081 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:14.082 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local test_state=started 2020-04-10T07:07:14.082 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_result 2020-04-10T07:07:14.082 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_state=stopped 2020-04-10T07:07:14.082 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2020-04-10T07:07:14.082 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ + local cluster_instance=cluster1 2020-04-10T07:07:14.082 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shiftgrep 2020-04-10T07:07:14.083 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: -i state+ 2020-04-10T07:07:14.083 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:local cluster=cluster1 2020-04-10T07:07:14.083 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance=cluster1 2020-04-10T07:07:14.083 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local loop_instance 2020-04-10T07:07:14.083 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0 2020-04-10T07:07:14.085 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ cluster1 != cluster1 ] 2020-04-10T07:07:14.085 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:07:14.086 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2020-04-10T07:07:14.086 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:07:14.086 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:14.086 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:14.086 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:07:14.086 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:14.087 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:14.087 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:14.087 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:14.087 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:14.087 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:14.087 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:14.088 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:14.088 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:14.088 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:14.088 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster1 2020-04-10T07:07:14.088 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:07:14.088 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:14.088 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:14.089 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:07:14.089 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:14.089 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:14.089 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:14.089 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:14.089 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:14.090 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:14.090 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:14.091 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:14.091 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:14.091 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:14.091 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:07:14.104 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:14.104 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:14.104 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:14.105 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2020-04-10T07:07:14.201 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:no valid command found; 1 closest matches: 2020-04-10T07:07:14.201 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status mirror/test1 2020-04-10T07:07:14.201 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:07:14.203 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2020-04-10T07:07:14.203 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:1 2020-04-10T07:07:14.203 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:14.203 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:14.203 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 cluster instance 2020-04-10T07:07:14.204 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:07:14.204 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:14.204 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:14.204 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:14.204 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:14.204 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:07:14.205 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:14.205 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:14.205 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:07:14.205 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:14.205 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:1 cluster1 2020-04-10T07:07:14.206 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:1 2020-04-10T07:07:14.206 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:14.206 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:14.206 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2020-04-10T07:07:14.206 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:07:14.206 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:14.207 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:14.207 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:14.207 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:14.207 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:07:14.207 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:14.207 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:14.207 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:07:14.208 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:14.208 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2020-04-10T07:07:14.221 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:14.222 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:14.222 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:14.222 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2020-04-10T07:07:14.317 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:14.318 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ status_result= "state": "Replaying" 2020-04-10T07:07:14.318 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep -i Replaying 2020-04-10T07:07:14.318 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo "state": "Replaying" 2020-04-10T07:07:14.319 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "state": "Replaying" 2020-04-10T07:07:14.320 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ current_state=started 2020-04-10T07:07:14.320 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test started = started 2020-04-10T07:07:14.320 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:14.320 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_started cluster1 mirror test1 2020-04-10T07:07:14.320 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:14.324 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:14.324 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:07:14.324 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_state cluster1 mirror test1 started 2020-04-10T07:07:14.325 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:14.325 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:14.325 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:07:14.325 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state=started 2020-04-10T07:07:14.325 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local s 2020-04-10T07:07:14.325 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:07:15.321 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_image_replay_state cluster1 mirror test1 started 2020-04-10T07:07:15.321 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:15.321 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:15.322 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:07:15.322 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local test_state=started 2020-04-10T07:07:15.322 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_result 2020-04-10T07:07:15.322 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_state=stopped 2020-04-10T07:07:15.322 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test1 2020-04-10T07:07:15.322 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_instance=cluster1 2020-04-10T07:07:15.322 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:15.323 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ + local cluster=cluster1 2020-04-10T07:07:15.323 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ localgrep instance=cluster1 -i 2020-04-10T07:07:15.323 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state 2020-04-10T07:07:15.323 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local loop_instance 2020-04-10T07:07:15.323 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0 2020-04-10T07:07:15.325 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ cluster1 != cluster1 ] 2020-04-10T07:07:15.325 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:07:15.325 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test1 2020-04-10T07:07:15.325 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:07:15.325 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:15.326 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:15.326 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:07:15.326 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:15.326 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:15.326 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:15.326 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:15.326 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:15.327 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:15.327 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:15.327 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:15.327 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:15.327 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:15.327 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster1 2020-04-10T07:07:15.327 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:07:15.328 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:15.328 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:15.328 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:07:15.328 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:15.328 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:15.328 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:15.328 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:15.329 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:15.329 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:15.330 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:15.330 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:15.330 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:15.330 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:15.330 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:07:15.344 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:15.345 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:15.345 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:15.345 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test1 2020-04-10T07:07:15.448 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:15.448 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ status_result= "state": "Replaying" 2020-04-10T07:07:15.449 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo "state": "Replaying" 2020-04-10T07:07:15.449 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep -i Replaying 2020-04-10T07:07:15.451 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "state": "Replaying" 2020-04-10T07:07:15.452 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ current_state=started 2020-04-10T07:07:15.452 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test started = started 2020-04-10T07:07:15.452 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:15.452 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2020-04-10T07:07:15.452 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:15.452 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:15.452 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:15.453 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:07:15.453 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:07:15.453 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:07:15.453 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:07:16.452 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying 2020-04-10T07:07:16.452 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:16.453 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:16.453 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:16.453 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:07:16.453 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:07:16.453 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:07:16.453 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror-test.mirror_status 2020-04-10T07:07:16.453 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror-test.mirror_status 2020-04-10T07:07:16.454 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:07:16.454 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:07:16.455 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test 2020-04-10T07:07:16.455 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ tee /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:07:16.518 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:test: 2020-04-10T07:07:16.519 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: global_id: 687e9f7d-ae46-4324-b8f7-7c16605e0fa6 2020-04-10T07:07:16.519 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+replaying 2020-04-10T07:07:16.519 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: replaying, {"replay_state": "idle", "remote_snapshot_timestamp": 1586502409, "local_snapshot_timestamp": 1586502409} 2020-04-10T07:07:16.519 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: service: mirror.1 on smithi204 2020-04-10T07:07:16.519 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:07:12 2020-04-10T07:07:16.526 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ state: .*up+replaying /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:07:16.528 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+replaying 2020-04-10T07:07:16.528 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ description: .* /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:07:16.529 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: replaying, {"replay_state": "idle", "remote_snapshot_timestamp": 1586502409, "local_snapshot_timestamp": 1586502409} 2020-04-10T07:07:16.530 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:07:16.530 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+replaying 2020-04-10T07:07:16.530 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^up+ 2020-04-10T07:07:16.531 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+replaying 2020-04-10T07:07:16.531 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep service: *mirror..* on /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:07:16.533 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: service: mirror.1 on smithi204 2020-04-10T07:07:16.533 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:07:16.536 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update=2020-04-10 07:07:12 2020-04-10T07:07:16.536 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying 2020-04-10 07:07:12 2020-04-10T07:07:16.536 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:16.536 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:16.536 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:16.537 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:07:16.537 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local prev_last_update=2020-04-10 07:07:12 2020-04-10T07:07:16.537 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror.mirror_status 2020-04-10T07:07:16.537 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror.mirror_status 2020-04-10T07:07:16.537 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:07:16.538 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror.mirror_status 2020-04-10T07:07:16.540 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2020-04-10T07:07:16.581 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: failed to query services: (13) Permission denied 2020-04-10T07:07:16.835 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update state 2020-04-10T07:07:16.835 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2020-04-10T07:07:16.840 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ last_update=2020-04-10 07:07:12 2020-04-10T07:07:16.840 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2020-04-10T07:07:16.845 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ state=up+replaying 2020-04-10T07:07:16.845 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+replaying 2020-04-10T07:07:16.846 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep up+replaying 2020-04-10T07:07:16.847 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+replaying 2020-04-10T07:07:16.847 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:16.847 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:16.847 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test1 up+replaying 2020-04-10T07:07:16.847 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:16.848 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:16.848 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:07:16.848 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:07:16.848 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:07:16.848 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:07:16.848 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:07:17.849 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_status_in_pool_dir cluster1 mirror test1 up+replaying 2020-04-10T07:07:17.849 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:17.849 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:17.849 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:07:17.849 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:07:17.850 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:07:17.850 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:07:17.850 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror-test1.mirror_status 2020-04-10T07:07:17.850 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror-test1.mirror_status 2020-04-10T07:07:17.850 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:07:17.851 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.mirror_status 2020-04-10T07:07:17.851 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test1 2020-04-10T07:07:17.852 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ tee /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.mirror_status 2020-04-10T07:07:17.929 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:test1: 2020-04-10T07:07:17.929 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: global_id: 56419286-13e9-49f6-a20d-2abe6ede083e 2020-04-10T07:07:17.929 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+replaying 2020-04-10T07:07:17.929 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: replaying, {"replay_state": "idle", "remote_snapshot_timestamp": 1586502402, "local_snapshot_timestamp": 1586502402} 2020-04-10T07:07:17.930 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: service: mirror.0 on smithi204 2020-04-10T07:07:17.930 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:07:12 2020-04-10T07:07:17.937 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ state: .*up+replaying /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.mirror_status 2020-04-10T07:07:17.939 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+replaying 2020-04-10T07:07:17.939 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ description: .* /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.mirror_status 2020-04-10T07:07:17.941 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: replaying, {"replay_state": "idle", "remote_snapshot_timestamp": 1586502402, "local_snapshot_timestamp": 1586502402} 2020-04-10T07:07:17.941 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:07:17.942 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+replaying 2020-04-10T07:07:17.942 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^up+ 2020-04-10T07:07:17.942 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+replaying 2020-04-10T07:07:17.943 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep service: *mirror..* on /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.mirror_status 2020-04-10T07:07:17.944 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: service: mirror.0 on smithi204 2020-04-10T07:07:17.945 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.mirror_status 2020-04-10T07:07:17.947 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update=2020-04-10 07:07:12 2020-04-10T07:07:17.947 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test1 up+replaying 2020-04-10 07:07:12 2020-04-10T07:07:17.948 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:17.948 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:17.948 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:07:17.948 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:07:17.948 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local prev_last_update=2020-04-10 07:07:12 2020-04-10T07:07:17.948 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror.mirror_status 2020-04-10T07:07:17.948 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror.mirror_status 2020-04-10T07:07:17.949 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:07:17.949 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror.mirror_status 2020-04-10T07:07:17.950 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2020-04-10T07:07:17.997 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: failed to query services: (13) Permission denied 2020-04-10T07:07:18.049 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update state 2020-04-10T07:07:18.050 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test1']/last_update 2020-04-10T07:07:18.055 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ last_update=2020-04-10 07:07:12 2020-04-10T07:07:18.055 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test1']/state 2020-04-10T07:07:18.060 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ state=up+replaying 2020-04-10T07:07:18.061 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+replaying 2020-04-10T07:07:18.061 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep up+replaying 2020-04-10T07:07:18.063 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+replaying 2020-04-10T07:07:18.063 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:18.063 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:18.063 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ all_admin_daemons cluster1 rbd mirror stop mirror cluster2-DNE 2020-04-10T07:07:18.063 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:18.064 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:18.064 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:07:18.066 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:0 rbd mirror stop mirror cluster2-DNE 2020-04-10T07:07:18.066 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:07:18.066 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:18.066 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:18.066 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:07:18.067 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:18.067 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:18.067 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:18.067 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:18.067 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:18.067 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:18.067 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:18.068 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:18.068 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:18.068 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:18.068 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster1 2020-04-10T07:07:18.068 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:07:18.068 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:18.068 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:18.069 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:07:18.069 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:18.069 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:18.069 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:18.069 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:18.069 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:18.069 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:18.069 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:18.070 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:18.070 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:18.070 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:18.070 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:07:18.086 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:18.087 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:18.087 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:18.087 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror stop mirror cluster2-DNE 2020-04-10T07:07:18.197 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:1 rbd mirror stop mirror cluster2-DNE 2020-04-10T07:07:18.197 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:1 2020-04-10T07:07:18.197 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:18.198 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:18.198 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 cluster instance 2020-04-10T07:07:18.198 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:07:18.198 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:18.198 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:18.198 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:18.198 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:18.199 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:07:18.199 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:18.199 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:18.199 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:07:18.199 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:18.199 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:1 cluster1 2020-04-10T07:07:18.199 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:1 2020-04-10T07:07:18.200 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:18.200 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:18.200 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2020-04-10T07:07:18.200 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:07:18.200 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:18.200 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:18.200 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:18.201 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:18.201 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:07:18.201 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:18.201 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:18.201 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:07:18.201 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:18.201 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2020-04-10T07:07:18.214 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:18.215 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:18.215 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:18.215 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror stop mirror cluster2-DNE 2020-04-10T07:07:18.314 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:2 rbd mirror stop mirror cluster2-DNE 2020-04-10T07:07:18.314 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:2 2020-04-10T07:07:18.314 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:18.315 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:18.315 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 cluster instance 2020-04-10T07:07:18.315 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:07:18.315 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:18.315 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:18.315 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:18.315 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:07:18.316 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:07:18.316 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:18.316 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:18.316 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:07:18.316 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:07:18.316 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:2 cluster1 2020-04-10T07:07:18.316 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:2 2020-04-10T07:07:18.317 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:18.317 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:18.317 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2020-04-10T07:07:18.317 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:07:18.317 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:18.317 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:18.317 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:18.318 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:07:18.318 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:07:18.318 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:18.318 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:18.318 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:07:18.318 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:07:18.318 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2020-04-10T07:07:18.327 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:07:18.327 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:07:18.327 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:07:18.327 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror stop mirror cluster2-DNE 2020-04-10T07:07:18.424 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:3 rbd mirror stop mirror cluster2-DNE 2020-04-10T07:07:18.424 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:3 2020-04-10T07:07:18.424 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:18.424 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:18.424 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:3 cluster instance 2020-04-10T07:07:18.424 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:3 2020-04-10T07:07:18.425 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:18.425 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:18.425 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:18.425 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:07:18.425 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 3 = cluster1:3 ] 2020-04-10T07:07:18.425 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:18.425 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:18.425 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=3 2020-04-10T07:07:18.426 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:07:18.426 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:3 cluster1 2020-04-10T07:07:18.426 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:3 2020-04-10T07:07:18.426 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:18.426 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:18.426 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:3 local_cluster instance 2020-04-10T07:07:18.426 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:3 2020-04-10T07:07:18.427 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:18.427 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:18.427 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:18.427 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:07:18.427 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 3 = cluster1:3 ] 2020-04-10T07:07:18.427 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:18.427 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:18.427 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=3 2020-04-10T07:07:18.428 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:07:18.428 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.3 admin socket 2020-04-10T07:07:18.443 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:07:18.443 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:07:18.444 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:07:18.444 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror stop mirror cluster2-DNE 2020-04-10T07:07:18.537 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_stopped cluster1 mirror test 2020-04-10T07:07:18.538 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:18.538 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:18.538 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:18.538 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_state cluster1 mirror test stopped 2020-04-10T07:07:18.538 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:18.538 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:18.539 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:18.539 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state=stopped 2020-04-10T07:07:18.539 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local s 2020-04-10T07:07:18.539 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:07:19.538 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_image_replay_state cluster1 mirror test stopped 2020-04-10T07:07:19.539 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:19.539 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:19.539 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:19.539 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local test_state=stopped 2020-04-10T07:07:19.539 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_result 2020-04-10T07:07:19.539 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_state=stopped 2020-04-10T07:07:19.540 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2020-04-10T07:07:19.541 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_instance=cluster1 2020-04-10T07:07:19.541 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift+ 2020-04-10T07:07:19.541 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:19.541 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ greplocal -i instance=cluster1 2020-04-10T07:07:19.541 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state 2020-04-10T07:07:19.541 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local loop_instance 2020-04-10T07:07:19.541 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0 2020-04-10T07:07:19.542 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ cluster1 != cluster1 ] 2020-04-10T07:07:19.542 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:07:19.543 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2020-04-10T07:07:19.543 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:07:19.544 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:19.544 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:19.544 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:07:19.544 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:19.544 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:19.544 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:19.545 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:19.545 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:19.545 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:19.545 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:19.545 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:19.545 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:19.545 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:19.545 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster1 2020-04-10T07:07:19.546 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:07:19.546 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:19.546 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:19.546 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:07:19.546 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:19.546 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:19.546 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:19.547 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:19.548 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:19.548 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:19.548 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:19.548 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:19.548 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:19.548 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:19.549 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:07:19.567 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:19.568 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:19.568 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:19.568 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2020-04-10T07:07:19.655 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:no valid command found; 1 closest matches: 2020-04-10T07:07:19.655 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status mirror/test1 2020-04-10T07:07:19.655 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:07:19.660 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2020-04-10T07:07:19.660 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:1 2020-04-10T07:07:19.660 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:19.660 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:19.660 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 cluster instance 2020-04-10T07:07:19.661 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:07:19.661 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:19.661 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:19.661 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:19.661 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:19.661 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:07:19.661 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:19.662 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:19.662 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:07:19.662 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:19.662 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:1 cluster1 2020-04-10T07:07:19.662 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:1 2020-04-10T07:07:19.662 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:19.662 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:19.663 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2020-04-10T07:07:19.663 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:07:19.663 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:19.663 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:19.663 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:19.663 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:19.663 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:07:19.663 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:19.664 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:19.664 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:07:19.664 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:19.664 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2020-04-10T07:07:19.677 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:19.677 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:19.677 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:19.677 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2020-04-10T07:07:19.772 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:19.773 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ status_result= "state": "Stopped" 2020-04-10T07:07:19.773 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo "state": "Stopped" 2020-04-10T07:07:19.773 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep -i Replaying 2020-04-10T07:07:19.775 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test stopped = stopped 2020-04-10T07:07:19.775 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:19.775 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_stopped cluster1 mirror test1 2020-04-10T07:07:19.775 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:19.775 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:19.775 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:07:19.775 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_state cluster1 mirror test1 stopped 2020-04-10T07:07:19.776 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:19.776 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:19.776 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:07:19.776 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state=stopped 2020-04-10T07:07:19.776 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local s 2020-04-10T07:07:19.776 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:07:20.776 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_image_replay_state cluster1 mirror test1 stopped 2020-04-10T07:07:20.776 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:20.776 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:20.777 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:07:20.777 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local test_state=stopped 2020-04-10T07:07:20.777 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_result 2020-04-10T07:07:20.777 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_state=stopped 2020-04-10T07:07:20.778 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test1 2020-04-10T07:07:20.778 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_instance=cluster1 2020-04-10T07:07:20.778 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:20.778 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ + local cluster=cluster1 2020-04-10T07:07:20.779 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ localgrep instance=cluster1 2020-04-10T07:07:20.779 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: -i state 2020-04-10T07:07:20.779 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local loop_instance 2020-04-10T07:07:20.779 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0 2020-04-10T07:07:20.779 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ cluster1 != cluster1 ] 2020-04-10T07:07:20.779 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:07:20.781 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test1 2020-04-10T07:07:20.781 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:07:20.781 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:20.782 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:20.782 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:07:20.782 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:20.782 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:20.782 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:20.782 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:20.782 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:20.782 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:20.783 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:20.783 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:20.783 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:20.783 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:20.783 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster1 2020-04-10T07:07:20.783 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:07:20.783 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:20.784 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:20.784 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:07:20.784 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:20.784 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:20.784 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:20.784 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:20.784 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:20.785 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:20.785 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:20.785 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:20.785 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:20.786 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:20.786 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:07:20.803 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:20.803 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:20.804 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:20.804 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test1 2020-04-10T07:07:20.905 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:20.906 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ status_result= "state": "Stopped" 2020-04-10T07:07:20.906 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo "state": "Stopped" 2020-04-10T07:07:20.906 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep -i Replaying 2020-04-10T07:07:20.908 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test stopped = stopped 2020-04-10T07:07:20.908 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:20.908 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+stopped 2020-04-10T07:07:20.908 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:20.908 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:20.909 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:20.909 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+stopped 2020-04-10T07:07:20.909 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:07:20.909 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:07:20.909 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:07:21.909 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_status_in_pool_dir cluster1 mirror test up+stopped 2020-04-10T07:07:21.909 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:21.910 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:21.910 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:21.910 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+stopped 2020-04-10T07:07:21.910 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:07:21.910 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:07:21.910 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror-test.mirror_status 2020-04-10T07:07:21.911 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror-test.mirror_status 2020-04-10T07:07:21.911 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:07:21.912 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:07:21.912 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test 2020-04-10T07:07:21.912 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ tee /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:07:21.986 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:test: 2020-04-10T07:07:21.986 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: global_id: 687e9f7d-ae46-4324-b8f7-7c16605e0fa6 2020-04-10T07:07:21.986 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+stopped 2020-04-10T07:07:21.986 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: stopped 2020-04-10T07:07:21.986 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: service: mirror.1 on smithi204 2020-04-10T07:07:21.987 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:07:18 2020-04-10T07:07:21.995 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ state: .*up+stopped /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:07:21.997 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+stopped 2020-04-10T07:07:21.997 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ description: .* /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:07:21.998 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: stopped 2020-04-10T07:07:21.999 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:07:21.999 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+stopped 2020-04-10T07:07:21.999 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^up+ 2020-04-10T07:07:22.000 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+stopped 2020-04-10T07:07:22.000 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep service: *mirror..* on /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:07:22.001 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: service: mirror.1 on smithi204 2020-04-10T07:07:22.001 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:07:22.004 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update=2020-04-10 07:07:18 2020-04-10T07:07:22.004 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+stopped 2020-04-10 07:07:18 2020-04-10T07:07:22.004 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:22.004 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:22.004 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:22.005 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+stopped 2020-04-10T07:07:22.005 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local prev_last_update=2020-04-10 07:07:18 2020-04-10T07:07:22.005 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror.mirror_status 2020-04-10T07:07:22.005 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror.mirror_status 2020-04-10T07:07:22.006 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:07:22.006 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror.mirror_status 2020-04-10T07:07:22.006 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2020-04-10T07:07:22.050 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: failed to query services: (13) Permission denied 2020-04-10T07:07:22.100 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update state 2020-04-10T07:07:22.100 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2020-04-10T07:07:22.106 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ last_update=2020-04-10 07:07:18 2020-04-10T07:07:22.106 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2020-04-10T07:07:22.111 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ state=up+stopped 2020-04-10T07:07:22.111 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+stopped 2020-04-10T07:07:22.112 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep up+stopped 2020-04-10T07:07:22.113 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+stopped 2020-04-10T07:07:22.113 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:22.113 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:22.113 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test1 up+stopped 2020-04-10T07:07:22.113 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:22.114 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:22.114 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:07:22.114 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+stopped 2020-04-10T07:07:22.114 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:07:22.114 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:07:22.114 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:07:23.115 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_status_in_pool_dir cluster1 mirror test1 up+stopped 2020-04-10T07:07:23.115 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:23.116 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:23.116 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:07:23.116 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+stopped 2020-04-10T07:07:23.116 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:07:23.117 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:07:23.117 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror-test1.mirror_status 2020-04-10T07:07:23.117 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror-test1.mirror_status 2020-04-10T07:07:23.117 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:07:23.119 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.mirror_status 2020-04-10T07:07:23.119 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test1 2020-04-10T07:07:23.119 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ tee /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.mirror_status 2020-04-10T07:07:23.188 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:test1: 2020-04-10T07:07:23.189 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: global_id: 56419286-13e9-49f6-a20d-2abe6ede083e 2020-04-10T07:07:23.189 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+stopped 2020-04-10T07:07:23.189 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: stopped 2020-04-10T07:07:23.189 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: service: mirror.0 on smithi204 2020-04-10T07:07:23.190 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:07:18 2020-04-10T07:07:23.196 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ state: .*up+stopped /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.mirror_status 2020-04-10T07:07:23.197 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+stopped 2020-04-10T07:07:23.198 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ description: .* /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.mirror_status 2020-04-10T07:07:23.199 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: stopped 2020-04-10T07:07:23.200 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:07:23.200 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+stopped 2020-04-10T07:07:23.200 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^up+ 2020-04-10T07:07:23.201 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+stopped 2020-04-10T07:07:23.201 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep service: *mirror..* on /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.mirror_status 2020-04-10T07:07:23.202 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: service: mirror.0 on smithi204 2020-04-10T07:07:23.202 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.mirror_status 2020-04-10T07:07:23.204 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update=2020-04-10 07:07:18 2020-04-10T07:07:23.205 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test1 up+stopped 2020-04-10 07:07:18 2020-04-10T07:07:23.205 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:23.205 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:23.205 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:07:23.206 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+stopped 2020-04-10T07:07:23.206 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local prev_last_update=2020-04-10 07:07:18 2020-04-10T07:07:23.206 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror.mirror_status 2020-04-10T07:07:23.206 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror.mirror_status 2020-04-10T07:07:23.207 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:07:23.207 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror.mirror_status 2020-04-10T07:07:23.208 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2020-04-10T07:07:23.248 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: failed to query services: (13) Permission denied 2020-04-10T07:07:23.296 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update state 2020-04-10T07:07:23.296 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test1']/last_update 2020-04-10T07:07:23.302 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ last_update=2020-04-10 07:07:18 2020-04-10T07:07:23.302 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test1']/state 2020-04-10T07:07:23.308 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ state=up+stopped 2020-04-10T07:07:23.308 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+stopped 2020-04-10T07:07:23.308 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep up+stopped 2020-04-10T07:07:23.316 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+stopped 2020-04-10T07:07:23.317 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:23.317 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:23.317 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemons cluster1 rbd mirror start mirror/test 2020-04-10T07:07:23.317 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_instance=cluster1 2020-04-10T07:07:23.317 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:23.318 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:23.318 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance=cluster1 2020-04-10T07:07:23.318 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local loop_instance 2020-04-10T07:07:23.318 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0 2020-04-10T07:07:23.319 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ cluster1 != cluster1 ] 2020-04-10T07:07:23.319 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:07:23.319 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:0 rbd mirror start mirror/test 2020-04-10T07:07:23.319 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:07:23.320 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:23.320 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:23.320 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:07:23.320 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:23.320 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:23.321 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:23.321 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:23.321 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:23.321 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:23.322 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:23.322 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:23.322 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:23.322 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:23.322 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster1 2020-04-10T07:07:23.323 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:07:23.323 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:23.323 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:23.323 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:07:23.324 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:23.324 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:23.324 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:23.324 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:23.326 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:23.326 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:23.327 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:23.327 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:23.327 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:23.327 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:23.328 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:07:23.335 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:23.335 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:23.336 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:23.336 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror start mirror/test 2020-04-10T07:07:23.422 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:no valid command found; 1 closest matches: 2020-04-10T07:07:23.422 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror start mirror/test1 2020-04-10T07:07:23.422 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:07:23.425 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:1 rbd mirror start mirror/test 2020-04-10T07:07:23.425 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:1 2020-04-10T07:07:23.425 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:23.425 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:23.425 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 cluster instance 2020-04-10T07:07:23.426 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:07:23.426 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:23.426 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:23.426 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:23.426 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:23.426 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:07:23.426 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:23.426 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:23.427 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:07:23.427 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:23.427 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:1 cluster1 2020-04-10T07:07:23.427 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:1 2020-04-10T07:07:23.427 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:23.427 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:23.427 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2020-04-10T07:07:23.428 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:07:23.428 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:23.428 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:23.428 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:23.428 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:23.428 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:07:23.428 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:23.428 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:23.429 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:07:23.429 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:23.429 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2020-04-10T07:07:23.444 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:23.444 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:23.445 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:23.445 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror start mirror/test 2020-04-10T07:07:23.550 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:23.550 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_started cluster1 mirror test 2020-04-10T07:07:23.550 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:23.551 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:23.551 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:23.551 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2020-04-10T07:07:23.551 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:23.551 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:23.551 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:23.551 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state=started 2020-04-10T07:07:23.552 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local s 2020-04-10T07:07:23.552 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:07:24.551 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_image_replay_state cluster1 mirror test started 2020-04-10T07:07:24.551 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:24.551 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:24.551 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:24.551 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local test_state=started 2020-04-10T07:07:24.551 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_result 2020-04-10T07:07:24.551 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_state=stopped 2020-04-10T07:07:24.552 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2020-04-10T07:07:24.553 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_instance=cluster1 2020-04-10T07:07:24.553 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:24.553 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ + local cluster=cluster1 2020-04-10T07:07:24.553 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance=cluster1 2020-04-10T07:07:24.553 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local loop_instance 2020-04-10T07:07:24.553 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:grep+ -isleep state 0 2020-04-10T07:07:24.553 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: 2020-04-10T07:07:24.553 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ cluster1 != cluster1 ] 2020-04-10T07:07:24.555 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:07:24.555 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2020-04-10T07:07:24.556 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:07:24.556 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:24.556 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:24.556 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:07:24.556 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:24.556 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:24.556 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:24.557 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:24.557 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:24.557 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:24.558 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:24.558 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:24.558 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:24.558 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:24.558 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster1 2020-04-10T07:07:24.558 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:07:24.558 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:24.559 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:24.559 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:07:24.559 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:24.559 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:24.559 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:24.559 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:24.559 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:24.560 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:24.560 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:24.560 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:24.561 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:24.561 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:24.561 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:07:24.577 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:24.577 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:24.578 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:24.578 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2020-04-10T07:07:24.671 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:no valid command found; 1 closest matches: 2020-04-10T07:07:24.672 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status mirror/test1 2020-04-10T07:07:24.672 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:07:24.674 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2020-04-10T07:07:24.674 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:1 2020-04-10T07:07:24.674 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:24.674 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:24.674 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 cluster instance 2020-04-10T07:07:24.674 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:07:24.674 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:24.675 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:24.675 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:24.675 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:24.675 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:07:24.675 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:24.675 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:24.675 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:07:24.676 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:24.676 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:1 cluster1 2020-04-10T07:07:24.676 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:1 2020-04-10T07:07:24.676 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:24.676 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:24.676 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2020-04-10T07:07:24.676 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:07:24.676 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:24.677 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:24.677 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:24.677 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:24.677 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:07:24.677 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:24.677 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:24.677 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:07:24.678 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:24.678 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2020-04-10T07:07:24.693 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:24.693 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:24.693 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:24.693 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2020-04-10T07:07:24.789 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:24.790 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ status_result= "state": "Replaying" 2020-04-10T07:07:24.790 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo "state": "Replaying" 2020-04-10T07:07:24.790 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep -i Replaying 2020-04-10T07:07:24.791 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "state": "Replaying" 2020-04-10T07:07:24.791 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ current_state=started 2020-04-10T07:07:24.792 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test started = started 2020-04-10T07:07:24.792 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:24.792 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2020-04-10T07:07:24.792 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:24.792 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:24.792 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:24.792 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:07:24.793 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:07:24.793 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:07:24.793 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:07:25.793 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying 2020-04-10T07:07:25.794 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:25.794 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:25.794 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:25.794 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:07:25.794 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:07:25.794 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:07:25.794 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror-test.mirror_status 2020-04-10T07:07:25.795 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror-test.mirror_status 2020-04-10T07:07:25.795 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:07:25.795 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:07:25.796 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test 2020-04-10T07:07:25.796 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ tee /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:07:25.872 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:test: 2020-04-10T07:07:25.872 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: global_id: 687e9f7d-ae46-4324-b8f7-7c16605e0fa6 2020-04-10T07:07:25.872 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+replaying 2020-04-10T07:07:25.873 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: replaying, {"replay_state": "idle", "remote_snapshot_timestamp": 1586502409, "local_snapshot_timestamp": 1586502409} 2020-04-10T07:07:25.873 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: service: mirror.1 on smithi204 2020-04-10T07:07:25.873 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:07:23 2020-04-10T07:07:25.880 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ state: .*up+replaying /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:07:25.882 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+replaying 2020-04-10T07:07:25.882 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ description: .* /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:07:25.883 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: replaying, {"replay_state": "idle", "remote_snapshot_timestamp": 1586502409, "local_snapshot_timestamp": 1586502409} 2020-04-10T07:07:25.884 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:07:25.884 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+replaying 2020-04-10T07:07:25.884 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^up+ 2020-04-10T07:07:25.885 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+replaying 2020-04-10T07:07:25.886 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep service: *mirror..* on /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:07:25.887 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: service: mirror.1 on smithi204 2020-04-10T07:07:25.887 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:07:25.890 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update=2020-04-10 07:07:23 2020-04-10T07:07:25.890 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying 2020-04-10 07:07:23 2020-04-10T07:07:25.890 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:25.890 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:25.890 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:25.891 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:07:25.891 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local prev_last_update=2020-04-10 07:07:23 2020-04-10T07:07:25.891 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror.mirror_status 2020-04-10T07:07:25.892 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror.mirror_status 2020-04-10T07:07:25.892 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:07:25.892 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror.mirror_status 2020-04-10T07:07:25.892 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2020-04-10T07:07:25.937 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: failed to query services: (13) Permission denied 2020-04-10T07:07:25.986 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update state 2020-04-10T07:07:25.986 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2020-04-10T07:07:25.991 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ last_update=2020-04-10 07:07:23 2020-04-10T07:07:25.992 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2020-04-10T07:07:25.997 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ state=up+replaying 2020-04-10T07:07:25.997 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ + echo up+replaying 2020-04-10T07:07:25.997 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:grep up+replaying 2020-04-10T07:07:25.998 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+replaying 2020-04-10T07:07:25.999 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:25.999 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:25.999 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ all_admin_daemons cluster1 rbd mirror start mirror cluster2-DNE 2020-04-10T07:07:25.999 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:26.000 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:26.000 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:07:26.002 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:0 rbd mirror start mirror cluster2-DNE 2020-04-10T07:07:26.002 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:07:26.002 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:26.002 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:26.002 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:07:26.002 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:26.002 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:26.003 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:26.003 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:26.003 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:26.003 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:26.003 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:26.003 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:26.003 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:26.004 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:26.004 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster1 2020-04-10T07:07:26.004 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:07:26.004 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:26.004 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:26.004 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:07:26.004 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:26.005 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:26.005 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:26.005 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:26.005 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:26.005 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:26.005 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:26.005 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:26.006 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:26.006 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:26.006 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:07:26.022 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:26.022 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:26.022 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:26.022 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror start mirror cluster2-DNE 2020-04-10T07:07:26.128 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:1 rbd mirror start mirror cluster2-DNE 2020-04-10T07:07:26.128 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:1 2020-04-10T07:07:26.128 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:26.128 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:26.128 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 cluster instance 2020-04-10T07:07:26.128 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:07:26.129 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:26.129 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:26.129 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:26.129 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:26.129 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:07:26.129 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:26.129 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:26.130 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:07:26.130 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:26.130 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:1 cluster1 2020-04-10T07:07:26.130 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:1 2020-04-10T07:07:26.130 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:26.130 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:26.130 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2020-04-10T07:07:26.131 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:07:26.131 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:26.131 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:26.131 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:26.131 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:26.131 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:07:26.131 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:26.132 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:26.132 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:07:26.132 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:26.132 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2020-04-10T07:07:26.149 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:26.150 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:26.150 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:26.150 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror start mirror cluster2-DNE 2020-04-10T07:07:26.243 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:2 rbd mirror start mirror cluster2-DNE 2020-04-10T07:07:26.243 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:2 2020-04-10T07:07:26.243 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:26.243 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:26.243 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 cluster instance 2020-04-10T07:07:26.244 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:07:26.244 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:26.244 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:26.244 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:26.244 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:07:26.244 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:07:26.244 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:26.245 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:26.245 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:07:26.245 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:07:26.245 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:2 cluster1 2020-04-10T07:07:26.245 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:2 2020-04-10T07:07:26.245 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:26.245 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:26.246 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2020-04-10T07:07:26.246 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:07:26.246 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:26.246 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:26.246 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:26.246 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:07:26.246 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:07:26.247 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:26.247 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:26.247 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:07:26.247 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:07:26.247 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2020-04-10T07:07:26.263 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:07:26.263 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:07:26.263 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:07:26.264 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror start mirror cluster2-DNE 2020-04-10T07:07:26.359 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:3 rbd mirror start mirror cluster2-DNE 2020-04-10T07:07:26.359 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:3 2020-04-10T07:07:26.359 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:26.359 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:26.359 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:3 cluster instance 2020-04-10T07:07:26.360 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:3 2020-04-10T07:07:26.360 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:26.360 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:26.360 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:26.360 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:07:26.360 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 3 = cluster1:3 ] 2020-04-10T07:07:26.360 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:26.361 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:26.361 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=3 2020-04-10T07:07:26.361 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:07:26.361 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:3 cluster1 2020-04-10T07:07:26.361 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:3 2020-04-10T07:07:26.361 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:26.361 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:26.362 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:3 local_cluster instance 2020-04-10T07:07:26.362 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:3 2020-04-10T07:07:26.363 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:26.363 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:26.363 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:26.364 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:07:26.364 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 3 = cluster1:3 ] 2020-04-10T07:07:26.364 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:26.364 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:26.364 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=3 2020-04-10T07:07:26.364 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:07:26.365 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.3 admin socket 2020-04-10T07:07:26.379 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:07:26.380 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:07:26.380 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:07:26.380 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror start mirror cluster2-DNE 2020-04-10T07:07:26.471 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_started cluster1 mirror test1 2020-04-10T07:07:26.471 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:26.472 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:26.472 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:07:26.472 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_state cluster1 mirror test1 started 2020-04-10T07:07:26.472 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:26.472 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:26.472 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:07:26.473 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state=started 2020-04-10T07:07:26.473 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local s 2020-04-10T07:07:26.473 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:07:27.472 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_image_replay_state cluster1 mirror test1 started 2020-04-10T07:07:27.472 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:27.472 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:27.473 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:07:27.473 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local test_state=started 2020-04-10T07:07:27.473 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_result 2020-04-10T07:07:27.473 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_state=stopped 2020-04-10T07:07:27.474 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test1 2020-04-10T07:07:27.474 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ + local cluster_instance=cluster1 2020-04-10T07:07:27.474 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shiftgrep 2020-04-10T07:07:27.474 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: -i state 2020-04-10T07:07:27.475 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:27.475 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance=cluster1 2020-04-10T07:07:27.475 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local loop_instance 2020-04-10T07:07:27.475 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0 2020-04-10T07:07:27.475 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ cluster1 != cluster1 ] 2020-04-10T07:07:27.475 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:07:27.477 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test1 2020-04-10T07:07:27.477 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:07:27.477 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:27.477 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:27.478 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:07:27.478 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:27.478 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:27.478 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:27.478 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:27.478 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:27.478 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:27.479 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:27.479 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:27.479 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:27.479 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:27.479 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster1 2020-04-10T07:07:27.479 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:07:27.479 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:27.479 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:27.480 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:07:27.480 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:27.480 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:27.480 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:27.480 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:27.480 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:27.480 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:27.481 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:27.481 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:27.481 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:27.482 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:27.482 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:07:27.500 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:27.500 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:27.500 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:27.500 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test1 2020-04-10T07:07:27.601 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:27.601 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ status_result= "state": "Replaying" 2020-04-10T07:07:27.602 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo "state": "Replaying" 2020-04-10T07:07:27.602 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep -i Replaying 2020-04-10T07:07:27.604 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "state": "Replaying" 2020-04-10T07:07:27.604 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ current_state=started 2020-04-10T07:07:27.604 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test started = started 2020-04-10T07:07:27.604 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:27.605 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test1 up+replaying 2020-04-10T07:07:27.605 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:27.605 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:27.605 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:07:27.605 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:07:27.605 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:07:27.605 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:07:27.606 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:07:28.605 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_status_in_pool_dir cluster1 mirror test1 up+replaying 2020-04-10T07:07:28.605 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:28.606 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:28.606 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:07:28.606 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:07:28.606 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:07:28.606 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:07:28.606 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror-test1.mirror_status 2020-04-10T07:07:28.606 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror-test1.mirror_status 2020-04-10T07:07:28.607 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:07:28.607 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.mirror_status 2020-04-10T07:07:28.608 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test1 2020-04-10T07:07:28.608 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ tee /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.mirror_status 2020-04-10T07:07:28.676 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:test1: 2020-04-10T07:07:28.677 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: global_id: 56419286-13e9-49f6-a20d-2abe6ede083e 2020-04-10T07:07:28.677 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+replaying 2020-04-10T07:07:28.677 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: replaying, {"replay_state": "idle", "remote_snapshot_timestamp": 1586502402, "local_snapshot_timestamp": 1586502402} 2020-04-10T07:07:28.677 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: service: mirror.0 on smithi204 2020-04-10T07:07:28.677 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:07:26 2020-04-10T07:07:28.684 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ state: .*up+replaying /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.mirror_status 2020-04-10T07:07:28.686 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+replaying 2020-04-10T07:07:28.686 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ description: .* /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.mirror_status 2020-04-10T07:07:28.688 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: replaying, {"replay_state": "idle", "remote_snapshot_timestamp": 1586502402, "local_snapshot_timestamp": 1586502402} 2020-04-10T07:07:28.688 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:07:28.689 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+replaying 2020-04-10T07:07:28.689 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^up+ 2020-04-10T07:07:28.690 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+replaying 2020-04-10T07:07:28.690 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep service: *mirror..* on /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.mirror_status 2020-04-10T07:07:28.691 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: service: mirror.0 on smithi204 2020-04-10T07:07:28.692 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.mirror_status 2020-04-10T07:07:28.694 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update=2020-04-10 07:07:26 2020-04-10T07:07:28.694 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test1 up+replaying 2020-04-10 07:07:26 2020-04-10T07:07:28.694 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:28.694 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:28.695 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:07:28.695 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:07:28.695 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local prev_last_update=2020-04-10 07:07:26 2020-04-10T07:07:28.695 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror.mirror_status 2020-04-10T07:07:28.696 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror.mirror_status 2020-04-10T07:07:28.696 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:07:28.697 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror.mirror_status 2020-04-10T07:07:28.697 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2020-04-10T07:07:28.741 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: failed to query services: (13) Permission denied 2020-04-10T07:07:28.789 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update state 2020-04-10T07:07:28.789 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test1']/last_update 2020-04-10T07:07:28.795 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ last_update=2020-04-10 07:07:26 2020-04-10T07:07:28.795 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test1']/state 2020-04-10T07:07:28.801 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ state=up+replaying 2020-04-10T07:07:28.801 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+replaying 2020-04-10T07:07:28.801 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep up+replaying 2020-04-10T07:07:28.802 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+replaying 2020-04-10T07:07:28.803 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:28.803 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:28.803 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemons cluster1 rbd mirror restart mirror/test 2020-04-10T07:07:28.803 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_instance=cluster1 2020-04-10T07:07:28.803 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:28.803 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:28.803 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance=cluster1 2020-04-10T07:07:28.804 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local loop_instance 2020-04-10T07:07:28.804 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0 2020-04-10T07:07:28.805 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ cluster1 != cluster1 ] 2020-04-10T07:07:28.806 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:07:28.806 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:0 rbd mirror restart mirror/test 2020-04-10T07:07:28.806 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:07:28.806 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:28.806 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:28.806 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:07:28.806 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:28.807 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:28.807 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:28.807 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:28.807 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:28.808 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:28.808 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:28.808 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:28.808 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:28.808 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:28.810 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster1 2020-04-10T07:07:28.810 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:07:28.810 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:28.810 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:28.810 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:07:28.810 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:28.811 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:28.811 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:28.811 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:28.811 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:28.813 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:28.814 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:28.814 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:28.814 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:28.814 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:28.814 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:07:28.827 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:28.828 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:28.828 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:28.828 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror restart mirror/test 2020-04-10T07:07:28.919 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:no valid command found; 1 closest matches: 2020-04-10T07:07:28.919 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror restart mirror/test1 2020-04-10T07:07:28.920 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:07:28.924 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:1 rbd mirror restart mirror/test 2020-04-10T07:07:28.924 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:1 2020-04-10T07:07:28.924 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:28.924 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:28.924 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 cluster instance 2020-04-10T07:07:28.924 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:07:28.925 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:28.925 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:28.925 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:28.925 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:28.925 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:07:28.925 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:28.925 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:28.926 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:07:28.926 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:28.926 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:1 cluster1 2020-04-10T07:07:28.926 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:1 2020-04-10T07:07:28.926 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:28.927 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:28.927 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2020-04-10T07:07:28.927 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:07:28.927 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:28.927 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:28.928 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:28.928 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:28.928 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:07:28.928 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:28.928 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:28.928 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:07:28.928 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:28.929 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2020-04-10T07:07:28.943 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:28.943 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:28.943 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:28.944 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror restart mirror/test 2020-04-10T07:07:29.044 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:29.044 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_started cluster1 mirror test 2020-04-10T07:07:29.044 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:29.044 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:29.045 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:29.045 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2020-04-10T07:07:29.045 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:29.045 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:29.045 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:29.045 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state=started 2020-04-10T07:07:29.045 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local s 2020-04-10T07:07:29.046 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:07:30.044 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_image_replay_state cluster1 mirror test started 2020-04-10T07:07:30.044 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:30.045 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:30.045 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:30.045 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local test_state=started 2020-04-10T07:07:30.045 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_result 2020-04-10T07:07:30.045 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_state=stopped 2020-04-10T07:07:30.053 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2020-04-10T07:07:30.054 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ + local cluster_instance=cluster1 2020-04-10T07:07:30.054 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:30.054 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:grep -i state 2020-04-10T07:07:30.054 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:30.054 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance=cluster1 2020-04-10T07:07:30.055 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local loop_instance 2020-04-10T07:07:30.055 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0 2020-04-10T07:07:30.055 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ cluster1 != cluster1 ] 2020-04-10T07:07:30.055 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:07:30.056 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2020-04-10T07:07:30.056 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:07:30.056 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:30.056 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:30.057 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:07:30.057 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:30.057 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:30.057 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:30.058 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:30.058 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:30.058 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:30.058 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:30.058 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:30.059 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:30.059 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:30.059 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster1 2020-04-10T07:07:30.059 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:07:30.060 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:30.060 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:30.061 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:07:30.061 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:30.061 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:30.061 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:30.061 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:30.062 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:30.062 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:30.062 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:30.062 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:30.063 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:30.063 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:30.063 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:07:30.070 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:30.070 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:30.070 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:30.070 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2020-04-10T07:07:30.164 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:no valid command found; 1 closest matches: 2020-04-10T07:07:30.164 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status mirror/test1 2020-04-10T07:07:30.165 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:07:30.169 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2020-04-10T07:07:30.169 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:1 2020-04-10T07:07:30.169 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:30.169 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:30.170 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 cluster instance 2020-04-10T07:07:30.170 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:07:30.170 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:30.170 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:30.171 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:30.171 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:30.171 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:07:30.171 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:30.171 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:30.172 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:07:30.172 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:30.172 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:1 cluster1 2020-04-10T07:07:30.172 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:1 2020-04-10T07:07:30.173 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:30.173 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:30.173 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2020-04-10T07:07:30.173 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:07:30.174 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:30.174 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:30.174 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:30.174 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:30.175 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:07:30.175 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:30.175 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:30.175 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:07:30.176 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:30.176 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2020-04-10T07:07:30.188 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:30.188 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:30.188 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:30.189 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2020-04-10T07:07:30.282 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:30.282 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ status_result= "state": "Replaying" 2020-04-10T07:07:30.283 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo "state": "Replaying" 2020-04-10T07:07:30.283 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep -i Replaying 2020-04-10T07:07:30.296 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "state": "Replaying" 2020-04-10T07:07:30.296 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ current_state=started 2020-04-10T07:07:30.296 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test started = started 2020-04-10T07:07:30.297 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:30.297 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2020-04-10T07:07:30.297 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:30.297 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:30.298 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:30.298 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:07:30.298 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:07:30.298 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:07:30.299 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:07:31.285 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying 2020-04-10T07:07:31.285 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:31.285 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:31.285 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:31.286 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:07:31.286 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:07:31.286 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:07:31.286 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror-test.mirror_status 2020-04-10T07:07:31.286 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror-test.mirror_status 2020-04-10T07:07:31.286 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:07:31.287 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:07:31.287 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test 2020-04-10T07:07:31.287 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ tee /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:07:31.360 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:test: 2020-04-10T07:07:31.361 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: global_id: 687e9f7d-ae46-4324-b8f7-7c16605e0fa6 2020-04-10T07:07:31.361 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+replaying 2020-04-10T07:07:31.361 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: replaying, {"replay_state": "idle", "remote_snapshot_timestamp": 1586502409, "local_snapshot_timestamp": 1586502409} 2020-04-10T07:07:31.361 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: service: mirror.1 on smithi204 2020-04-10T07:07:31.361 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:07:29 2020-04-10T07:07:31.368 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ state: .*up+replaying /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:07:31.370 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+replaying 2020-04-10T07:07:31.370 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ description: .* /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:07:31.372 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: replaying, {"replay_state": "idle", "remote_snapshot_timestamp": 1586502409, "local_snapshot_timestamp": 1586502409} 2020-04-10T07:07:31.372 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:07:31.372 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ + echo up+replaying 2020-04-10T07:07:31.372 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:grep ^up+ 2020-04-10T07:07:31.373 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+replaying 2020-04-10T07:07:31.373 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep service: *mirror..* on /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:07:31.375 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: service: mirror.1 on smithi204 2020-04-10T07:07:31.376 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:07:31.378 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update=2020-04-10 07:07:29 2020-04-10T07:07:31.378 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying 2020-04-10 07:07:29 2020-04-10T07:07:31.379 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:31.379 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:31.379 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:31.379 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:07:31.379 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local prev_last_update=2020-04-10 07:07:29 2020-04-10T07:07:31.379 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror.mirror_status 2020-04-10T07:07:31.380 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror.mirror_status 2020-04-10T07:07:31.380 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:07:31.380 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror.mirror_status 2020-04-10T07:07:31.381 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2020-04-10T07:07:31.423 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: failed to query services: (13) Permission denied 2020-04-10T07:07:31.473 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update state 2020-04-10T07:07:31.473 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2020-04-10T07:07:31.479 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ last_update=2020-04-10 07:07:29 2020-04-10T07:07:31.479 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2020-04-10T07:07:31.486 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ state=up+replaying 2020-04-10T07:07:31.486 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ + echo up+replaying 2020-04-10T07:07:31.486 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:grep up+replaying 2020-04-10T07:07:31.490 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+replaying 2020-04-10T07:07:31.490 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:31.490 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:31.491 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ all_admin_daemons cluster1 rbd mirror restart mirror cluster2-DNE 2020-04-10T07:07:31.491 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:31.491 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:31.491 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:07:31.491 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:0 rbd mirror restart mirror cluster2-DNE 2020-04-10T07:07:31.492 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:07:31.492 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:31.492 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:31.492 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:07:31.492 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:31.493 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:31.493 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:31.493 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:31.493 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:31.494 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:31.494 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:31.494 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:31.494 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:31.494 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:31.495 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster1 2020-04-10T07:07:31.495 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:07:31.495 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:31.495 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:31.495 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:07:31.496 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:31.496 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:31.496 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:31.496 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:31.498 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:31.498 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:31.498 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:31.498 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:31.498 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:31.499 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:31.499 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:07:31.510 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:31.510 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:31.510 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:31.510 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror restart mirror cluster2-DNE 2020-04-10T07:07:31.608 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:1 rbd mirror restart mirror cluster2-DNE 2020-04-10T07:07:31.609 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:1 2020-04-10T07:07:31.609 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:31.609 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:31.609 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 cluster instance 2020-04-10T07:07:31.609 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:07:31.609 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:31.609 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:31.610 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:31.610 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:31.610 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:07:31.610 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:31.610 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:31.610 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:07:31.610 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:31.611 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:1 cluster1 2020-04-10T07:07:31.611 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:1 2020-04-10T07:07:31.611 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:31.611 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:31.611 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2020-04-10T07:07:31.611 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:07:31.611 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:31.612 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:31.612 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:31.612 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:31.612 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:07:31.612 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:31.612 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:31.612 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:07:31.613 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:31.613 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2020-04-10T07:07:31.621 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:31.621 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:31.621 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:31.621 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror restart mirror cluster2-DNE 2020-04-10T07:07:31.715 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:2 rbd mirror restart mirror cluster2-DNE 2020-04-10T07:07:31.716 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:2 2020-04-10T07:07:31.716 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:31.716 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:31.716 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 cluster instance 2020-04-10T07:07:31.716 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:07:31.716 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:31.716 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:31.717 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:31.717 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:07:31.717 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:07:31.717 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:31.717 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:31.717 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:07:31.717 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:07:31.718 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:2 cluster1 2020-04-10T07:07:31.718 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:2 2020-04-10T07:07:31.718 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:31.718 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:31.718 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2020-04-10T07:07:31.718 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:07:31.718 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:31.718 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:31.719 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:31.719 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:07:31.719 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:07:31.719 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:31.719 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:31.719 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:07:31.719 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:07:31.720 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2020-04-10T07:07:31.740 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:07:31.740 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:07:31.740 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:07:31.741 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror restart mirror cluster2-DNE 2020-04-10T07:07:31.834 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:3 rbd mirror restart mirror cluster2-DNE 2020-04-10T07:07:31.834 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:3 2020-04-10T07:07:31.834 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:31.834 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:31.834 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:3 cluster instance 2020-04-10T07:07:31.835 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:3 2020-04-10T07:07:31.835 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:31.835 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:31.835 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:31.835 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:07:31.835 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 3 = cluster1:3 ] 2020-04-10T07:07:31.835 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:31.836 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:31.836 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=3 2020-04-10T07:07:31.836 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:07:31.836 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:3 cluster1 2020-04-10T07:07:31.836 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:3 2020-04-10T07:07:31.836 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:31.836 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:31.837 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:3 local_cluster instance 2020-04-10T07:07:31.837 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:3 2020-04-10T07:07:31.837 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:31.837 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:31.837 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:31.837 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:07:31.837 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 3 = cluster1:3 ] 2020-04-10T07:07:31.838 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:31.838 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:31.838 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=3 2020-04-10T07:07:31.838 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:07:31.838 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.3 admin socket 2020-04-10T07:07:31.853 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:07:31.853 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:07:31.853 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:07:31.853 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror restart mirror cluster2-DNE 2020-04-10T07:07:31.943 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_started cluster1 mirror test 2020-04-10T07:07:31.943 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:31.944 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:31.944 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:31.944 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2020-04-10T07:07:31.944 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:31.944 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:31.945 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:31.945 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state=started 2020-04-10T07:07:31.945 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local s 2020-04-10T07:07:31.945 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:07:32.944 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_image_replay_state cluster1 mirror test started 2020-04-10T07:07:32.944 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:32.944 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:32.944 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:32.944 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local test_state=started 2020-04-10T07:07:32.945 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_result 2020-04-10T07:07:32.945 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_state=stopped 2020-04-10T07:07:32.946 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2020-04-10T07:07:32.946 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_instance=cluster1 2020-04-10T07:07:32.946 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:32.946 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ + local cluster=cluster1 2020-04-10T07:07:32.946 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance=cluster1 2020-04-10T07:07:32.947 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:grep+ -ilocal state loop_instance 2020-04-10T07:07:32.947 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: 2020-04-10T07:07:32.947 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0 2020-04-10T07:07:32.947 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ cluster1 != cluster1 ] 2020-04-10T07:07:32.947 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:07:32.949 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2020-04-10T07:07:32.949 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:07:32.949 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:32.949 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:32.949 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:07:32.949 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:32.949 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:32.950 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:32.950 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:32.950 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:32.950 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:32.950 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:32.950 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:32.950 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:32.950 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:32.956 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster1 2020-04-10T07:07:32.957 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:07:32.957 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:32.958 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:32.958 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:07:32.958 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:32.959 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:32.959 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:32.959 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:32.959 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:32.962 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:32.962 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:32.963 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:32.963 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:32.963 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:32.963 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:07:32.968 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:32.969 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:32.969 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:32.969 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2020-04-10T07:07:33.071 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:no valid command found; 1 closest matches: 2020-04-10T07:07:33.074 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status mirror/test1 2020-04-10T07:07:33.074 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:07:33.074 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2020-04-10T07:07:33.075 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:1 2020-04-10T07:07:33.075 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:33.075 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:33.075 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 cluster instance 2020-04-10T07:07:33.075 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:07:33.075 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:33.075 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:33.076 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:33.076 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:33.076 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:07:33.076 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:33.076 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:33.076 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:07:33.076 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:33.077 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:1 cluster1 2020-04-10T07:07:33.077 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:1 2020-04-10T07:07:33.077 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:33.077 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:33.077 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2020-04-10T07:07:33.077 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:07:33.078 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:33.078 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:33.078 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:33.078 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:33.078 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:07:33.078 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:33.078 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:33.078 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:07:33.079 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:33.079 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2020-04-10T07:07:33.084 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:33.085 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:33.085 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:33.085 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2020-04-10T07:07:33.177 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:33.178 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ status_result= "state": "Replaying" 2020-04-10T07:07:33.178 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo "state": "Replaying" 2020-04-10T07:07:33.179 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep -i Replaying 2020-04-10T07:07:33.182 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "state": "Replaying" 2020-04-10T07:07:33.182 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ current_state=started 2020-04-10T07:07:33.182 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test started = started 2020-04-10T07:07:33.183 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:33.183 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_started cluster1 mirror test1 2020-04-10T07:07:33.183 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:33.183 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:33.183 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:07:33.183 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_state cluster1 mirror test1 started 2020-04-10T07:07:33.183 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:33.184 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:33.184 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:07:33.184 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state=started 2020-04-10T07:07:33.184 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local s 2020-04-10T07:07:33.184 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:07:34.181 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_image_replay_state cluster1 mirror test1 started 2020-04-10T07:07:34.181 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:34.181 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:34.182 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:07:34.182 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local test_state=started 2020-04-10T07:07:34.182 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_result 2020-04-10T07:07:34.182 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_state=stopped 2020-04-10T07:07:34.183 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test1 2020-04-10T07:07:34.183 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ + local cluster_instance=cluster1 2020-04-10T07:07:34.183 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:34.183 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:grep -i state 2020-04-10T07:07:34.184 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:34.184 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance=cluster1 2020-04-10T07:07:34.184 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local loop_instance 2020-04-10T07:07:34.184 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0 2020-04-10T07:07:34.184 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ cluster1 != cluster1 ] 2020-04-10T07:07:34.184 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:07:34.186 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test1 2020-04-10T07:07:34.187 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:07:34.187 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:34.187 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:34.187 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:07:34.187 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:34.187 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:34.187 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:34.188 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:34.188 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:34.188 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:34.188 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:34.188 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:34.188 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:34.188 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:34.188 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster1 2020-04-10T07:07:34.189 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:07:34.189 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:34.189 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:34.189 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:07:34.189 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:34.189 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:34.189 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:34.190 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:34.190 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:34.190 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:34.190 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:34.190 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:34.190 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:34.190 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:34.190 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:07:34.208 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:34.208 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:34.208 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:34.208 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test1 2020-04-10T07:07:34.311 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:34.312 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ status_result= "state": "Replaying" 2020-04-10T07:07:34.312 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo "state": "Replaying" 2020-04-10T07:07:34.312 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep -i Replaying 2020-04-10T07:07:34.314 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "state": "Replaying" 2020-04-10T07:07:34.315 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ current_state=started 2020-04-10T07:07:34.315 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test started = started 2020-04-10T07:07:34.315 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:34.315 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ all_admin_daemons cluster1 rbd mirror stop mirror cluster2-DNE 2020-04-10T07:07:34.315 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:34.315 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:34.315 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:07:34.319 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:0 rbd mirror stop mirror cluster2-DNE 2020-04-10T07:07:34.320 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:07:34.320 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:34.320 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:34.320 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:07:34.320 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:34.320 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:34.320 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:34.321 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:34.321 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:34.321 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:34.321 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:34.321 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:34.321 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:34.321 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:34.322 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster1 2020-04-10T07:07:34.322 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:07:34.322 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:34.322 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:34.322 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:07:34.322 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:34.322 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:34.323 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:34.323 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:34.323 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:34.323 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:34.323 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:34.323 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:34.323 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:34.323 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:34.324 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:07:34.331 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:34.331 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:34.331 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:34.331 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror stop mirror cluster2-DNE 2020-04-10T07:07:34.434 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:1 rbd mirror stop mirror cluster2-DNE 2020-04-10T07:07:34.434 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:1 2020-04-10T07:07:34.434 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:34.434 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:34.434 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 cluster instance 2020-04-10T07:07:34.435 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:07:34.435 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:34.435 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:34.435 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:34.435 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:34.435 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:07:34.435 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:34.436 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:34.436 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:07:34.436 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:34.436 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:1 cluster1 2020-04-10T07:07:34.436 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:1 2020-04-10T07:07:34.436 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:34.436 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:34.437 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2020-04-10T07:07:34.437 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:07:34.437 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:34.437 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:34.437 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:34.437 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:34.438 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:07:34.438 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:34.438 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:34.438 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:07:34.438 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:34.438 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2020-04-10T07:07:34.452 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:34.452 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:34.452 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:34.452 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror stop mirror cluster2-DNE 2020-04-10T07:07:34.548 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:2 rbd mirror stop mirror cluster2-DNE 2020-04-10T07:07:34.548 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:2 2020-04-10T07:07:34.548 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:34.549 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:34.549 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 cluster instance 2020-04-10T07:07:34.549 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:07:34.549 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:34.550 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:34.551 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:34.551 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:07:34.552 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:07:34.552 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:34.552 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:34.552 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:07:34.553 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:07:34.554 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:2 cluster1 2020-04-10T07:07:34.554 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:2 2020-04-10T07:07:34.554 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:34.554 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:34.555 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2020-04-10T07:07:34.555 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:07:34.555 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:34.556 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:34.556 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:34.556 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:07:34.556 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:07:34.557 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:34.558 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:34.558 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:07:34.558 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:07:34.558 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2020-04-10T07:07:34.561 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:07:34.561 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:07:34.562 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:07:34.562 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror stop mirror cluster2-DNE 2020-04-10T07:07:34.654 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:3 rbd mirror stop mirror cluster2-DNE 2020-04-10T07:07:34.654 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:3 2020-04-10T07:07:34.655 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:34.655 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:34.655 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:3 cluster instance 2020-04-10T07:07:34.655 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:3 2020-04-10T07:07:34.655 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:34.655 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:34.655 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:34.656 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:07:34.656 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 3 = cluster1:3 ] 2020-04-10T07:07:34.656 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:34.656 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:34.656 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=3 2020-04-10T07:07:34.656 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:07:34.656 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:3 cluster1 2020-04-10T07:07:34.657 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:3 2020-04-10T07:07:34.657 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:34.657 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:34.657 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:3 local_cluster instance 2020-04-10T07:07:34.657 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:3 2020-04-10T07:07:34.657 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:34.657 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:34.658 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:34.658 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:07:34.658 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 3 = cluster1:3 ] 2020-04-10T07:07:34.658 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:34.658 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:34.658 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=3 2020-04-10T07:07:34.658 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:07:34.659 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.3 admin socket 2020-04-10T07:07:34.673 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:07:34.673 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:07:34.673 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:07:34.673 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror stop mirror cluster2-DNE 2020-04-10T07:07:34.777 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_stopped cluster1 mirror test 2020-04-10T07:07:34.777 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:34.778 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:34.778 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:34.778 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_state cluster1 mirror test stopped 2020-04-10T07:07:34.778 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:34.778 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:34.778 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:34.778 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state=stopped 2020-04-10T07:07:34.778 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local s 2020-04-10T07:07:34.779 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:07:35.781 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_image_replay_state cluster1 mirror test stopped 2020-04-10T07:07:35.781 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:35.781 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:35.781 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:35.781 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local test_state=stopped 2020-04-10T07:07:35.782 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_result 2020-04-10T07:07:35.782 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_state=stopped 2020-04-10T07:07:35.782 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2020-04-10T07:07:35.782 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_instance=cluster1 2020-04-10T07:07:35.782 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:35.782 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local+ cluster=cluster1 2020-04-10T07:07:35.782 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance=cluster1 2020-04-10T07:07:35.783 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local loop_instance 2020-04-10T07:07:35.783 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:grep -i state 2020-04-10T07:07:35.783 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0 2020-04-10T07:07:35.783 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ cluster1 != cluster1 ] 2020-04-10T07:07:35.783 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:07:35.787 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2020-04-10T07:07:35.787 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:07:35.787 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:35.787 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:35.787 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:07:35.787 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:35.787 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:35.788 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:35.788 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:35.788 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:35.788 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:35.788 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:35.788 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:35.788 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:35.789 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:35.789 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster1 2020-04-10T07:07:35.789 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:07:35.789 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:35.789 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:35.789 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:07:35.789 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:35.790 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:35.790 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:35.790 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:35.790 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:35.790 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:35.790 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:35.790 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:35.791 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:35.791 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:35.791 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:07:35.807 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:35.807 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:35.807 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:35.807 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2020-04-10T07:07:35.901 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:no valid command found; 1 closest matches: 2020-04-10T07:07:35.901 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status mirror/test1 2020-04-10T07:07:35.901 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:07:35.903 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2020-04-10T07:07:35.903 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:1 2020-04-10T07:07:35.904 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:35.904 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:35.904 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 cluster instance 2020-04-10T07:07:35.904 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:07:35.904 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:35.904 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:35.904 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:35.905 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:35.905 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:07:35.905 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:35.905 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:35.905 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:07:35.905 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:35.905 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:1 cluster1 2020-04-10T07:07:35.905 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:1 2020-04-10T07:07:35.906 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:35.906 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:35.906 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2020-04-10T07:07:35.906 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:07:35.906 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:35.906 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:35.906 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:35.907 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:35.907 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:07:35.907 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:35.907 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:35.907 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:07:35.907 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:35.907 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2020-04-10T07:07:35.929 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:35.929 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:35.929 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:35.929 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2020-04-10T07:07:36.021 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:36.022 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ status_result= "state": "Stopped" 2020-04-10T07:07:36.022 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo "state": "Stopped" 2020-04-10T07:07:36.022 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep -i Replaying 2020-04-10T07:07:36.023 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test stopped = stopped 2020-04-10T07:07:36.023 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:36.023 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_stopped cluster1 mirror test1 2020-04-10T07:07:36.024 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:36.024 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:36.024 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:07:36.024 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_state cluster1 mirror test1 stopped 2020-04-10T07:07:36.024 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:36.024 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:36.024 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:07:36.025 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state=stopped 2020-04-10T07:07:36.025 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local s 2020-04-10T07:07:36.025 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:07:37.024 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_image_replay_state cluster1 mirror test1 stopped 2020-04-10T07:07:37.024 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:37.025 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:37.025 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:07:37.025 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local test_state=stopped 2020-04-10T07:07:37.025 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_result 2020-04-10T07:07:37.025 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_state=stopped 2020-04-10T07:07:37.026 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test1 2020-04-10T07:07:37.026 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ + localgrep cluster_instance=cluster1 -i 2020-04-10T07:07:37.027 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state 2020-04-10T07:07:37.027 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:37.027 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:37.027 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance=cluster1 2020-04-10T07:07:37.027 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local loop_instance 2020-04-10T07:07:37.027 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0 2020-04-10T07:07:37.027 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ cluster1 != cluster1 ] 2020-04-10T07:07:37.027 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:07:37.029 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test1 2020-04-10T07:07:37.029 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:07:37.029 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:37.030 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:37.030 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:07:37.030 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:37.030 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:37.030 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:37.030 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:37.030 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:37.031 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:37.031 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:37.031 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:37.031 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:37.031 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:37.031 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster1 2020-04-10T07:07:37.031 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:07:37.032 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:37.032 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:37.032 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:07:37.032 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:37.032 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:37.032 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:37.033 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:37.033 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:37.034 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:37.034 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:37.038 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:37.039 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:37.039 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:37.039 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:07:37.051 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:37.051 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:37.051 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:37.051 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test1 2020-04-10T07:07:37.163 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:37.164 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ status_result= "state": "Stopped" 2020-04-10T07:07:37.164 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo "state": "Stopped" 2020-04-10T07:07:37.164 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep -i Replaying 2020-04-10T07:07:37.165 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test stopped = stopped 2020-04-10T07:07:37.165 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:37.165 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+stopped 2020-04-10T07:07:37.165 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:37.166 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:37.166 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:37.166 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+stopped 2020-04-10T07:07:37.166 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:07:37.166 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:07:37.166 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:07:38.167 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_status_in_pool_dir cluster1 mirror test up+stopped 2020-04-10T07:07:38.167 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:38.167 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:38.167 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:38.167 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+stopped 2020-04-10T07:07:38.168 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:07:38.168 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:07:38.168 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror-test.mirror_status 2020-04-10T07:07:38.168 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror-test.mirror_status 2020-04-10T07:07:38.168 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:07:38.169 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:07:38.169 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ tee /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:07:38.169 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test 2020-04-10T07:07:38.242 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:test: 2020-04-10T07:07:38.242 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: global_id: 687e9f7d-ae46-4324-b8f7-7c16605e0fa6 2020-04-10T07:07:38.242 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+stopped 2020-04-10T07:07:38.242 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: stopped 2020-04-10T07:07:38.243 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: service: mirror.1 on smithi204 2020-04-10T07:07:38.243 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:07:34 2020-04-10T07:07:38.252 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ state: .*up+stopped /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:07:38.252 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+stopped 2020-04-10T07:07:38.253 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ description: .* /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:07:38.260 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: stopped 2020-04-10T07:07:38.260 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+stopped 2020-04-10T07:07:38.260 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: service: mirror.1 on smithi204 2020-04-10T07:07:38.261 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:07:38.261 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ + grep ^up+ 2020-04-10T07:07:38.261 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:echo up+stopped 2020-04-10T07:07:38.261 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep service: *mirror..* on /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:07:38.263 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:07:38.263 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update=2020-04-10 07:07:34 2020-04-10T07:07:38.263 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+stopped 2020-04-10 07:07:34 2020-04-10T07:07:38.263 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:38.263 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:38.263 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:38.263 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+stopped 2020-04-10T07:07:38.264 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local prev_last_update=2020-04-10 07:07:34 2020-04-10T07:07:38.264 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror.mirror_status 2020-04-10T07:07:38.264 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror.mirror_status 2020-04-10T07:07:38.264 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:07:38.264 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror.mirror_status 2020-04-10T07:07:38.264 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2020-04-10T07:07:38.307 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: failed to query services: (13) Permission denied 2020-04-10T07:07:38.354 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update state 2020-04-10T07:07:38.354 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2020-04-10T07:07:38.360 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ last_update=2020-04-10 07:07:34 2020-04-10T07:07:38.360 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2020-04-10T07:07:38.366 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ state=up+stopped 2020-04-10T07:07:38.366 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ + echo up+stopped 2020-04-10T07:07:38.366 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:grep up+stopped 2020-04-10T07:07:38.367 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+stopped 2020-04-10T07:07:38.367 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:38.368 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:38.368 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test1 up+stopped 2020-04-10T07:07:38.368 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:38.368 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:38.368 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:07:38.368 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+stopped 2020-04-10T07:07:38.368 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:07:38.369 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:07:38.369 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:07:39.374 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_status_in_pool_dir cluster1 mirror test1 up+stopped 2020-04-10T07:07:39.374 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:39.375 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:39.375 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:07:39.375 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+stopped 2020-04-10T07:07:39.375 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:07:39.375 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:07:39.375 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror-test1.mirror_status 2020-04-10T07:07:39.376 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror-test1.mirror_status 2020-04-10T07:07:39.376 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:07:39.376 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.mirror_status 2020-04-10T07:07:39.376 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test1 2020-04-10T07:07:39.376 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ tee /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.mirror_status 2020-04-10T07:07:39.434 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:test1: 2020-04-10T07:07:39.434 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: global_id: 56419286-13e9-49f6-a20d-2abe6ede083e 2020-04-10T07:07:39.434 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+stopped 2020-04-10T07:07:39.434 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: stopped 2020-04-10T07:07:39.434 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: service: mirror.0 on smithi204 2020-04-10T07:07:39.435 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:07:34 2020-04-10T07:07:39.441 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ state: .*up+stopped /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.mirror_status 2020-04-10T07:07:39.444 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+stopped 2020-04-10T07:07:39.444 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ description: .* /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.mirror_status 2020-04-10T07:07:39.446 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: stopped 2020-04-10T07:07:39.446 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:07:39.447 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+stopped 2020-04-10T07:07:39.447 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^up+ 2020-04-10T07:07:39.448 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+stopped 2020-04-10T07:07:39.448 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep service: *mirror..* on /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.mirror_status 2020-04-10T07:07:39.449 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: service: mirror.0 on smithi204 2020-04-10T07:07:39.449 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.mirror_status 2020-04-10T07:07:39.451 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update=2020-04-10 07:07:34 2020-04-10T07:07:39.451 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test1 up+stopped 2020-04-10 07:07:34 2020-04-10T07:07:39.451 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:39.452 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:39.452 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:07:39.452 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+stopped 2020-04-10T07:07:39.452 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local prev_last_update=2020-04-10 07:07:34 2020-04-10T07:07:39.452 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror.mirror_status 2020-04-10T07:07:39.452 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror.mirror_status 2020-04-10T07:07:39.452 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:07:39.453 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror.mirror_status 2020-04-10T07:07:39.453 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2020-04-10T07:07:39.493 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: failed to query services: (13) Permission denied 2020-04-10T07:07:39.533 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update state 2020-04-10T07:07:39.534 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test1']/last_update 2020-04-10T07:07:39.541 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ last_update=2020-04-10 07:07:34 2020-04-10T07:07:39.541 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test1']/state 2020-04-10T07:07:39.541 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ state=up+stopped 2020-04-10T07:07:39.541 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+stopped 2020-04-10T07:07:39.547 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep up+stopped 2020-04-10T07:07:39.549 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+stopped 2020-04-10T07:07:39.550 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:39.550 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:39.550 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ all_admin_daemons cluster1 rbd mirror restart mirror cluster2-DNE 2020-04-10T07:07:39.550 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:39.550 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:39.550 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:07:39.550 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:0 rbd mirror restart mirror cluster2-DNE 2020-04-10T07:07:39.551 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:07:39.551 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:39.551 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:39.551 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:07:39.551 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:39.551 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:39.551 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:39.552 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:39.552 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:39.552 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:39.552 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:39.552 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:39.552 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:39.552 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:39.553 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster1 2020-04-10T07:07:39.553 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:07:39.553 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:39.553 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:39.553 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:07:39.553 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:39.553 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:39.554 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:39.554 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:39.554 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:39.555 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:39.555 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:39.555 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:39.555 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:39.555 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:39.555 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:07:39.563 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:39.563 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:39.563 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:39.563 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror restart mirror cluster2-DNE 2020-04-10T07:07:39.672 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:1 rbd mirror restart mirror cluster2-DNE 2020-04-10T07:07:39.672 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:1 2020-04-10T07:07:39.672 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:39.672 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:39.672 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 cluster instance 2020-04-10T07:07:39.673 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:07:39.673 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:39.673 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:39.673 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:39.673 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:39.673 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:07:39.673 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:39.673 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:39.674 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:07:39.674 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:39.674 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:1 cluster1 2020-04-10T07:07:39.674 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:1 2020-04-10T07:07:39.674 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:39.674 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:39.674 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2020-04-10T07:07:39.675 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:07:39.675 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:39.675 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:39.675 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:39.675 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:39.675 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:07:39.675 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:39.676 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:39.676 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:07:39.676 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:39.676 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2020-04-10T07:07:39.687 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:39.687 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:39.687 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:39.687 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror restart mirror cluster2-DNE 2020-04-10T07:07:39.786 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:2 rbd mirror restart mirror cluster2-DNE 2020-04-10T07:07:39.786 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:2 2020-04-10T07:07:39.786 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:39.787 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:39.787 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 cluster instance 2020-04-10T07:07:39.787 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:07:39.787 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:39.787 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:39.787 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:39.787 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:07:39.788 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:07:39.788 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:39.788 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:39.788 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:07:39.788 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:07:39.788 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:2 cluster1 2020-04-10T07:07:39.788 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:2 2020-04-10T07:07:39.789 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:39.789 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:39.789 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2020-04-10T07:07:39.789 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:07:39.789 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:39.789 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:39.789 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:39.790 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:07:39.790 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:07:39.790 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:39.790 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:39.790 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:07:39.790 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:07:39.790 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2020-04-10T07:07:39.805 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:07:39.805 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:07:39.805 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:07:39.805 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror restart mirror cluster2-DNE 2020-04-10T07:07:39.895 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:3 rbd mirror restart mirror cluster2-DNE 2020-04-10T07:07:39.895 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:3 2020-04-10T07:07:39.895 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:39.895 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:39.896 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:3 cluster instance 2020-04-10T07:07:39.896 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:3 2020-04-10T07:07:39.896 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:39.896 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:39.896 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:39.896 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:07:39.896 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 3 = cluster1:3 ] 2020-04-10T07:07:39.897 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:39.897 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:39.897 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=3 2020-04-10T07:07:39.897 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:07:39.897 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:3 cluster1 2020-04-10T07:07:39.897 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:3 2020-04-10T07:07:39.897 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:39.898 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:39.898 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:3 local_cluster instance 2020-04-10T07:07:39.898 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:3 2020-04-10T07:07:39.898 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:39.898 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:39.898 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:39.898 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:07:39.898 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 3 = cluster1:3 ] 2020-04-10T07:07:39.899 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:39.899 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:39.899 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=3 2020-04-10T07:07:39.899 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:07:39.899 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.3 admin socket 2020-04-10T07:07:39.917 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:07:39.917 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:07:39.917 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:07:39.917 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror restart mirror cluster2-DNE 2020-04-10T07:07:40.012 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_started cluster1 mirror test 2020-04-10T07:07:40.012 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:40.012 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:40.012 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:40.012 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2020-04-10T07:07:40.012 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:40.013 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:40.013 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:40.013 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state=started 2020-04-10T07:07:40.013 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local s 2020-04-10T07:07:40.013 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:07:41.013 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_image_replay_state cluster1 mirror test started 2020-04-10T07:07:41.013 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:41.013 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:41.013 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:41.014 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local test_state=started 2020-04-10T07:07:41.014 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_result 2020-04-10T07:07:41.014 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_state=stopped 2020-04-10T07:07:41.015 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2020-04-10T07:07:41.015 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_instance=cluster1 2020-04-10T07:07:41.015 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ + shift 2020-04-10T07:07:41.015 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1grep 2020-04-10T07:07:41.016 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: -i state 2020-04-10T07:07:41.016 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance=cluster1 2020-04-10T07:07:41.016 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local loop_instance 2020-04-10T07:07:41.016 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0 2020-04-10T07:07:41.016 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ cluster1 != cluster1 ] 2020-04-10T07:07:41.018 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:07:41.019 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2020-04-10T07:07:41.019 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:07:41.019 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:41.019 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:41.019 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:07:41.019 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:41.020 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:41.020 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:41.020 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:41.020 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:41.020 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:41.020 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:41.020 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:41.021 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:41.021 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:41.021 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster1 2020-04-10T07:07:41.021 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:07:41.021 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:41.021 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:41.021 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:07:41.021 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:41.022 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:41.022 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:41.022 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:41.022 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:41.022 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:41.022 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:41.022 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:41.023 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:41.023 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:41.023 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:07:41.042 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:41.042 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:41.042 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:41.042 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2020-04-10T07:07:41.128 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:no valid command found; 1 closest matches: 2020-04-10T07:07:41.128 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status mirror/test1 2020-04-10T07:07:41.128 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:07:41.134 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2020-04-10T07:07:41.134 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:1 2020-04-10T07:07:41.134 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:41.134 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:41.134 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 cluster instance 2020-04-10T07:07:41.135 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:07:41.135 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:41.135 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:41.135 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:41.135 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:41.135 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:07:41.135 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:41.136 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:41.136 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:07:41.136 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:41.136 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:1 cluster1 2020-04-10T07:07:41.136 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:1 2020-04-10T07:07:41.136 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:41.136 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:41.137 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2020-04-10T07:07:41.137 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:07:41.137 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:41.137 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:41.137 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:41.137 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:41.137 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:07:41.138 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:41.138 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:41.138 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:07:41.138 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:41.138 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2020-04-10T07:07:41.152 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:41.152 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:41.152 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:41.153 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2020-04-10T07:07:41.246 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:41.246 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ status_result= "state": "Replaying" 2020-04-10T07:07:41.246 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ + echo "state": "Replaying" 2020-04-10T07:07:41.247 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:grep -i Replaying 2020-04-10T07:07:41.249 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "state": "Replaying" 2020-04-10T07:07:41.250 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ current_state=started 2020-04-10T07:07:41.250 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test started = started 2020-04-10T07:07:41.250 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:41.250 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_started cluster1 mirror test1 2020-04-10T07:07:41.250 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:41.250 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:41.251 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:07:41.251 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_state cluster1 mirror test1 started 2020-04-10T07:07:41.251 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:41.251 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:41.251 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:07:41.251 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state=started 2020-04-10T07:07:41.251 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local s 2020-04-10T07:07:41.251 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:07:42.249 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_image_replay_state cluster1 mirror test1 started 2020-04-10T07:07:42.249 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:42.250 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:42.250 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:07:42.250 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local test_state=started 2020-04-10T07:07:42.250 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_result 2020-04-10T07:07:42.250 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_state=stopped 2020-04-10T07:07:42.252 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test1 2020-04-10T07:07:42.252 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ + local cluster_instance=cluster1 2020-04-10T07:07:42.252 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:42.252 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:grep -i state 2020-04-10T07:07:42.252 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:42.252 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance=cluster1 2020-04-10T07:07:42.253 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local loop_instance 2020-04-10T07:07:42.253 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0 2020-04-10T07:07:42.253 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ cluster1 != cluster1 ] 2020-04-10T07:07:42.253 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:07:42.256 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test1 2020-04-10T07:07:42.256 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:07:42.256 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:42.256 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:42.257 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:07:42.257 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:42.257 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:42.258 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:42.258 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:42.258 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:42.258 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:42.258 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:42.258 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:42.258 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:42.259 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:42.259 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster1 2020-04-10T07:07:42.259 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:07:42.259 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:42.259 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:42.259 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:07:42.259 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:42.260 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:42.260 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:42.260 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:42.260 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:42.260 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:42.260 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:42.260 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:42.261 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:42.261 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:42.261 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:07:42.276 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:42.277 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:42.277 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:42.277 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test1 2020-04-10T07:07:42.381 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:42.381 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ status_result= "state": "Replaying" 2020-04-10T07:07:42.382 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo "state": "Replaying" 2020-04-10T07:07:42.382 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep -i Replaying 2020-04-10T07:07:42.383 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "state": "Replaying" 2020-04-10T07:07:42.384 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ current_state=started 2020-04-10T07:07:42.384 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test started = started 2020-04-10T07:07:42.384 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:42.385 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2020-04-10T07:07:42.385 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:42.385 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:42.385 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:42.385 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:07:42.385 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:07:42.385 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:07:42.385 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:07:43.386 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying 2020-04-10T07:07:43.386 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:43.387 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:43.387 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:43.387 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:07:43.387 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:07:43.387 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:07:43.387 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror-test.mirror_status 2020-04-10T07:07:43.388 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror-test.mirror_status 2020-04-10T07:07:43.388 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:07:43.388 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:07:43.389 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test 2020-04-10T07:07:43.389 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ tee /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:07:43.458 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:test: 2020-04-10T07:07:43.459 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: global_id: 687e9f7d-ae46-4324-b8f7-7c16605e0fa6 2020-04-10T07:07:43.459 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+replaying 2020-04-10T07:07:43.459 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: replaying, {"replay_state": "idle", "remote_snapshot_timestamp": 1586502409, "local_snapshot_timestamp": 1586502409} 2020-04-10T07:07:43.459 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: service: mirror.1 on smithi204 2020-04-10T07:07:43.459 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:07:39 2020-04-10T07:07:43.467 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ state: .*up+replaying /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:07:43.469 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+replaying 2020-04-10T07:07:43.469 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ description: .* /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:07:43.471 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: replaying, {"replay_state": "idle", "remote_snapshot_timestamp": 1586502409, "local_snapshot_timestamp": 1586502409} 2020-04-10T07:07:43.472 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:07:43.472 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+replaying 2020-04-10T07:07:43.472 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^up+ 2020-04-10T07:07:43.473 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+replaying 2020-04-10T07:07:43.473 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep service: *mirror..* on /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:07:43.476 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: service: mirror.1 on smithi204 2020-04-10T07:07:43.476 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:07:43.477 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update=2020-04-10 07:07:39 2020-04-10T07:07:43.477 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying 2020-04-10 07:07:39 2020-04-10T07:07:43.477 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:43.478 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:43.478 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:43.478 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:07:43.478 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local prev_last_update=2020-04-10 07:07:39 2020-04-10T07:07:43.478 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror.mirror_status 2020-04-10T07:07:43.479 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror.mirror_status 2020-04-10T07:07:43.479 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:07:43.480 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror.mirror_status 2020-04-10T07:07:43.480 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2020-04-10T07:07:43.523 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: failed to query services: (13) Permission denied 2020-04-10T07:07:43.576 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update state 2020-04-10T07:07:43.576 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2020-04-10T07:07:43.582 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ last_update=2020-04-10 07:07:39 2020-04-10T07:07:43.582 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2020-04-10T07:07:43.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ state=up+replaying 2020-04-10T07:07:43.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+replaying 2020-04-10T07:07:43.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep up+replaying 2020-04-10T07:07:43.590 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:43.590 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:43.590 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test1 up+replaying 2020-04-10T07:07:43.590 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:43.590 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:43.590 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:07:43.590 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:07:43.591 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:07:43.591 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:07:43.591 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:07:43.591 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+replaying 2020-04-10T07:07:44.590 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_status_in_pool_dir cluster1 mirror test1 up+replaying 2020-04-10T07:07:44.591 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:44.591 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:44.591 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:07:44.591 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:07:44.591 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:07:44.591 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:07:44.592 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror-test1.mirror_status 2020-04-10T07:07:44.592 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror-test1.mirror_status 2020-04-10T07:07:44.592 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:07:44.592 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.mirror_status 2020-04-10T07:07:44.593 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test1 2020-04-10T07:07:44.593 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ tee /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.mirror_status 2020-04-10T07:07:44.660 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:test1: 2020-04-10T07:07:44.660 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: global_id: 56419286-13e9-49f6-a20d-2abe6ede083e 2020-04-10T07:07:44.661 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+replaying 2020-04-10T07:07:44.661 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: replaying, {"replay_state": "idle", "remote_snapshot_timestamp": 1586502402, "local_snapshot_timestamp": 1586502402} 2020-04-10T07:07:44.661 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: service: mirror.0 on smithi204 2020-04-10T07:07:44.661 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:07:39 2020-04-10T07:07:44.668 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ state: .*up+replaying /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.mirror_status 2020-04-10T07:07:44.670 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+replaying 2020-04-10T07:07:44.670 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ description: .* /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.mirror_status 2020-04-10T07:07:44.672 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: replaying, {"replay_state": "idle", "remote_snapshot_timestamp": 1586502402, "local_snapshot_timestamp": 1586502402} 2020-04-10T07:07:44.672 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:07:44.673 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+replaying 2020-04-10T07:07:44.673 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^up+ 2020-04-10T07:07:44.674 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+replaying 2020-04-10T07:07:44.674 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep service: *mirror..* on /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.mirror_status 2020-04-10T07:07:44.675 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: service: mirror.0 on smithi204 2020-04-10T07:07:44.676 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test1.mirror_status 2020-04-10T07:07:44.678 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update=2020-04-10 07:07:39 2020-04-10T07:07:44.679 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test1 up+replaying 2020-04-10 07:07:39 2020-04-10T07:07:44.679 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:44.679 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:44.679 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test1 2020-04-10T07:07:44.679 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:07:44.679 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local prev_last_update=2020-04-10 07:07:39 2020-04-10T07:07:44.679 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror.mirror_status 2020-04-10T07:07:44.680 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror.mirror_status 2020-04-10T07:07:44.680 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:07:44.680 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror.mirror_status 2020-04-10T07:07:44.681 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2020-04-10T07:07:44.723 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: failed to query services: (13) Permission denied 2020-04-10T07:07:44.774 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update state 2020-04-10T07:07:44.774 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test1']/last_update 2020-04-10T07:07:44.778 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ last_update=2020-04-10 07:07:39 2020-04-10T07:07:44.779 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test1']/state 2020-04-10T07:07:44.782 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ state=up+replaying 2020-04-10T07:07:44.782 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+replaying 2020-04-10T07:07:44.782 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep up+replaying 2020-04-10T07:07:44.784 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+replaying 2020-04-10T07:07:44.784 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:44.784 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:44.784 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ flush cluster1 2020-04-10T07:07:44.784 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:44.785 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool= 2020-04-10T07:07:44.785 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image= 2020-04-10T07:07:44.785 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cmd=rbd mirror flush 2020-04-10T07:07:44.785 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:07:44.785 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemons cluster1 rbd mirror flush 2020-04-10T07:07:44.785 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_instance=cluster1 2020-04-10T07:07:44.786 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:44.786 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:44.786 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance=cluster1 2020-04-10T07:07:44.786 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local loop_instance 2020-04-10T07:07:44.786 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0 2020-04-10T07:07:44.801 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ cluster1 != cluster1 ] 2020-04-10T07:07:44.801 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:07:44.801 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:0 rbd mirror flush 2020-04-10T07:07:44.802 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:07:44.802 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:44.802 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:44.802 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:07:44.802 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:44.802 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:44.803 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:44.803 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:44.810 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:44.810 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:44.810 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:44.810 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:44.810 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:44.810 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:44.811 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster1 2020-04-10T07:07:44.811 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:07:44.811 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:44.811 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:44.811 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:07:44.811 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:44.812 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:44.812 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:44.812 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:44.812 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:44.812 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:44.812 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:44.813 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:44.813 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:44.813 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:44.813 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:07:44.814 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:44.814 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:44.814 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:44.814 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush 2020-04-10T07:07:44.916 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:44.916 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ all_admin_daemons cluster1 rbd mirror status 2020-04-10T07:07:44.916 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:44.916 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:44.916 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:07:44.919 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:0 rbd mirror status 2020-04-10T07:07:44.920 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:07:44.920 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:44.920 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:44.920 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:07:44.920 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:44.920 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:44.921 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:44.921 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:44.921 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:44.921 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:44.921 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:44.921 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:44.922 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:44.922 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:44.922 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster1 2020-04-10T07:07:44.922 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:07:44.922 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:44.922 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:44.922 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:07:44.923 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:44.923 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:44.923 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:44.923 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:44.923 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:44.923 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:44.924 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:44.924 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:44.924 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:44.924 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:44.924 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:07:44.940 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:44.940 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:44.940 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:44.940 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status 2020-04-10T07:07:45.034 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:{ 2020-04-10T07:07:45.034 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "pool_replayers": [ 2020-04-10T07:07:45.034 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: { 2020-04-10T07:07:45.034 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "peer": "uuid: 2571f49c-d767-4112-ae12-d8b71263d58b cluster: cluster2-DNE client: client.mirror", 2020-04-10T07:07:45.034 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "pool": "mirror", 2020-04-10T07:07:45.035 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "instance_id": "4415", 2020-04-10T07:07:45.035 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "state": "stopped (manual)", 2020-04-10T07:07:45.035 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "leader_instance_id": "4415", 2020-04-10T07:07:45.035 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "leader": true, 2020-04-10T07:07:45.035 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "instances": [ 2020-04-10T07:07:45.035 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "4415", 2020-04-10T07:07:45.036 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "4419", 2020-04-10T07:07:45.036 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "4427", 2020-04-10T07:07:45.036 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "4433" 2020-04-10T07:07:45.036 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: ], 2020-04-10T07:07:45.036 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "local_cluster_admin_socket": "/var/run/ceph/client.mirror.0.15642.cluster1.94084521538664.asok", 2020-04-10T07:07:45.036 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "remote_cluster_admin_socket": "/var/run/ceph/client.mirror.15642.cluster2-DNE.94084534195304.asok", 2020-04-10T07:07:45.036 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "sync_throttler": { 2020-04-10T07:07:45.037 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "max_parallel_requests": 5, 2020-04-10T07:07:45.037 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "running_requests": 0, 2020-04-10T07:07:45.037 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "waiting_requests": 0 2020-04-10T07:07:45.037 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: }, 2020-04-10T07:07:45.037 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "deletion_throttler": { 2020-04-10T07:07:45.037 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "max_parallel_requests": 1, 2020-04-10T07:07:45.038 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "running_requests": 0, 2020-04-10T07:07:45.038 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "waiting_requests": 0 2020-04-10T07:07:45.038 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: }, 2020-04-10T07:07:45.038 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "image_replayers": [ 2020-04-10T07:07:45.038 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: { 2020-04-10T07:07:45.038 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "name": "mirror/test1", 2020-04-10T07:07:45.038 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "state": "Replaying" 2020-04-10T07:07:45.039 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: } 2020-04-10T07:07:45.039 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: ], 2020-04-10T07:07:45.039 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "image_deleter": { 2020-04-10T07:07:45.039 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "image_deleter_status": { 2020-04-10T07:07:45.039 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "delete_images_queue": [], 2020-04-10T07:07:45.039 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "failed_deletes_queue": [] 2020-04-10T07:07:45.039 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: } 2020-04-10T07:07:45.040 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: }, 2020-04-10T07:07:45.040 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "namespaces": [ 2020-04-10T07:07:45.040 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: { 2020-04-10T07:07:45.040 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "name": "ns1", 2020-04-10T07:07:45.040 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "image_replayers": [], 2020-04-10T07:07:45.040 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "image_deleter": { 2020-04-10T07:07:45.040 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "image_deleter_status": { 2020-04-10T07:07:45.041 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "delete_images_queue": [], 2020-04-10T07:07:45.041 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "failed_deletes_queue": [] 2020-04-10T07:07:45.041 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: } 2020-04-10T07:07:45.041 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: } 2020-04-10T07:07:45.041 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: }, 2020-04-10T07:07:45.041 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: { 2020-04-10T07:07:45.041 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "name": "ns2", 2020-04-10T07:07:45.042 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "image_replayers": [], 2020-04-10T07:07:45.042 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "image_deleter": { 2020-04-10T07:07:45.042 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "image_deleter_status": { 2020-04-10T07:07:45.042 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "delete_images_queue": [], 2020-04-10T07:07:45.042 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "failed_deletes_queue": [] 2020-04-10T07:07:45.042 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: } 2020-04-10T07:07:45.043 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: } 2020-04-10T07:07:45.043 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: } 2020-04-10T07:07:45.043 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: ] 2020-04-10T07:07:45.043 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: }, 2020-04-10T07:07:45.043 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: { 2020-04-10T07:07:45.043 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "peer": "uuid: 467a3104-15d9-4d31-90d5-8005aa7c9eeb cluster: cluster2-DNE client: client.mirror", 2020-04-10T07:07:45.043 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "pool": "mirror_parent", 2020-04-10T07:07:45.044 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "instance_id": "4436", 2020-04-10T07:07:45.044 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "state": "stopped (manual)", 2020-04-10T07:07:45.044 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "leader_instance_id": "4436", 2020-04-10T07:07:45.044 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "leader": true, 2020-04-10T07:07:45.044 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "instances": [ 2020-04-10T07:07:45.044 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "4436", 2020-04-10T07:07:45.044 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "4439", 2020-04-10T07:07:45.045 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "4443", 2020-04-10T07:07:45.045 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "4447" 2020-04-10T07:07:45.045 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: ], 2020-04-10T07:07:45.045 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "local_cluster_admin_socket": "/var/run/ceph/client.mirror.0.15642.cluster1.94084547155048.asok", 2020-04-10T07:07:45.045 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "remote_cluster_admin_socket": "/var/run/ceph/client.mirror.15642.cluster2-DNE.94084559582312.asok", 2020-04-10T07:07:45.045 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "sync_throttler": { 2020-04-10T07:07:45.045 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "max_parallel_requests": 5, 2020-04-10T07:07:45.046 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "running_requests": 0, 2020-04-10T07:07:45.046 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "waiting_requests": 0 2020-04-10T07:07:45.046 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: }, 2020-04-10T07:07:45.046 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "deletion_throttler": { 2020-04-10T07:07:45.046 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "max_parallel_requests": 1, 2020-04-10T07:07:45.046 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "running_requests": 0, 2020-04-10T07:07:45.046 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "waiting_requests": 0 2020-04-10T07:07:45.047 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: }, 2020-04-10T07:07:45.047 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "image_replayers": [], 2020-04-10T07:07:45.047 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "image_deleter": { 2020-04-10T07:07:45.047 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "image_deleter_status": { 2020-04-10T07:07:45.047 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "delete_images_queue": [], 2020-04-10T07:07:45.047 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "failed_deletes_queue": [] 2020-04-10T07:07:45.048 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: } 2020-04-10T07:07:45.048 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: }, 2020-04-10T07:07:45.048 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "namespaces": [] 2020-04-10T07:07:45.048 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: } 2020-04-10T07:07:45.048 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: ] 2020-04-10T07:07:45.048 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:} 2020-04-10T07:07:45.051 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:1 rbd mirror status 2020-04-10T07:07:45.051 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:1 2020-04-10T07:07:45.051 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:45.052 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:45.052 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 cluster instance 2020-04-10T07:07:45.052 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:07:45.052 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:45.052 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:45.052 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:45.052 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:45.053 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:07:45.053 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:45.053 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:45.053 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:07:45.053 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:45.053 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:1 cluster1 2020-04-10T07:07:45.053 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:1 2020-04-10T07:07:45.054 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:45.054 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:45.054 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2020-04-10T07:07:45.054 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:07:45.054 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:45.054 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:45.055 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:45.055 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:45.055 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:07:45.055 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:45.055 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:45.055 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:07:45.055 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:45.056 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2020-04-10T07:07:45.066 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:45.066 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:45.067 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:45.067 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status 2020-04-10T07:07:45.156 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:{ 2020-04-10T07:07:45.156 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "pool_replayers": [ 2020-04-10T07:07:45.156 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: { 2020-04-10T07:07:45.156 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "peer": "uuid: 2571f49c-d767-4112-ae12-d8b71263d58b cluster: cluster2-DNE client: client.mirror", 2020-04-10T07:07:45.157 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "pool": "mirror", 2020-04-10T07:07:45.157 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "instance_id": "4419", 2020-04-10T07:07:45.157 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "state": "stopped (manual)", 2020-04-10T07:07:45.157 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "leader_instance_id": "4415", 2020-04-10T07:07:45.157 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "leader": false, 2020-04-10T07:07:45.157 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "local_cluster_admin_socket": "/var/run/ceph/client.mirror.1.15673.cluster1.94072177701992.asok", 2020-04-10T07:07:45.158 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "remote_cluster_admin_socket": "/var/run/ceph/client.mirror.15673.cluster2-DNE.94072190432360.asok", 2020-04-10T07:07:45.158 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "sync_throttler": { 2020-04-10T07:07:45.158 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "max_parallel_requests": 5, 2020-04-10T07:07:45.158 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "running_requests": 0, 2020-04-10T07:07:45.158 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "waiting_requests": 0 2020-04-10T07:07:45.158 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: }, 2020-04-10T07:07:45.158 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "deletion_throttler": { 2020-04-10T07:07:45.159 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "max_parallel_requests": 1, 2020-04-10T07:07:45.159 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "running_requests": 0, 2020-04-10T07:07:45.159 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "waiting_requests": 0 2020-04-10T07:07:45.159 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: }, 2020-04-10T07:07:45.159 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "image_replayers": [ 2020-04-10T07:07:45.159 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: { 2020-04-10T07:07:45.159 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "name": "mirror/test", 2020-04-10T07:07:45.160 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "state": "Replaying" 2020-04-10T07:07:45.160 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: } 2020-04-10T07:07:45.160 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: ], 2020-04-10T07:07:45.160 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "namespaces": [ 2020-04-10T07:07:45.160 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: { 2020-04-10T07:07:45.160 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "name": "ns1", 2020-04-10T07:07:45.161 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "image_replayers": [] 2020-04-10T07:07:45.161 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: }, 2020-04-10T07:07:45.161 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: { 2020-04-10T07:07:45.161 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "name": "ns2", 2020-04-10T07:07:45.161 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "image_replayers": [] 2020-04-10T07:07:45.161 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: } 2020-04-10T07:07:45.161 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: ] 2020-04-10T07:07:45.162 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: }, 2020-04-10T07:07:45.162 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: { 2020-04-10T07:07:45.162 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "peer": "uuid: 467a3104-15d9-4d31-90d5-8005aa7c9eeb cluster: cluster2-DNE client: client.mirror", 2020-04-10T07:07:45.162 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "pool": "mirror_parent", 2020-04-10T07:07:45.162 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "instance_id": "4439", 2020-04-10T07:07:45.162 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "state": "stopped (manual)", 2020-04-10T07:07:45.163 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "leader_instance_id": "4436", 2020-04-10T07:07:45.163 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "leader": false, 2020-04-10T07:07:45.163 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "local_cluster_admin_socket": "/var/run/ceph/client.mirror.1.15673.cluster1.94072203375720.asok", 2020-04-10T07:07:45.163 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "remote_cluster_admin_socket": "/var/run/ceph/client.mirror.15673.cluster2-DNE.94072216065128.asok", 2020-04-10T07:07:45.163 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "sync_throttler": { 2020-04-10T07:07:45.163 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "max_parallel_requests": 5, 2020-04-10T07:07:45.163 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "running_requests": 0, 2020-04-10T07:07:45.164 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "waiting_requests": 0 2020-04-10T07:07:45.164 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: }, 2020-04-10T07:07:45.164 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "deletion_throttler": { 2020-04-10T07:07:45.164 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "max_parallel_requests": 1, 2020-04-10T07:07:45.164 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "running_requests": 0, 2020-04-10T07:07:45.164 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "waiting_requests": 0 2020-04-10T07:07:45.164 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: }, 2020-04-10T07:07:45.165 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "image_replayers": [], 2020-04-10T07:07:45.165 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "namespaces": [] 2020-04-10T07:07:45.165 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: } 2020-04-10T07:07:45.165 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: ] 2020-04-10T07:07:45.165 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:} 2020-04-10T07:07:45.167 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:2 rbd mirror status 2020-04-10T07:07:45.167 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:2 2020-04-10T07:07:45.167 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:45.168 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:45.168 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 cluster instance 2020-04-10T07:07:45.168 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:07:45.168 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:45.168 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:45.168 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:45.169 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:07:45.169 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:07:45.169 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:45.169 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:45.169 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:07:45.169 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:07:45.169 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:2 cluster1 2020-04-10T07:07:45.170 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:2 2020-04-10T07:07:45.170 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:45.170 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:45.170 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2020-04-10T07:07:45.170 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:07:45.170 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:45.170 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:45.171 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:45.171 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:07:45.171 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:07:45.171 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:45.171 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:45.171 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:07:45.171 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:07:45.172 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2020-04-10T07:07:45.187 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:07:45.187 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:07:45.187 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:07:45.188 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status 2020-04-10T07:07:45.269 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:{ 2020-04-10T07:07:45.269 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "pool_replayers": [ 2020-04-10T07:07:45.270 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: { 2020-04-10T07:07:45.270 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "peer": "uuid: 2571f49c-d767-4112-ae12-d8b71263d58b cluster: cluster2-DNE client: client.mirror", 2020-04-10T07:07:45.270 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "pool": "mirror", 2020-04-10T07:07:45.270 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "instance_id": "4427", 2020-04-10T07:07:45.270 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "state": "stopped (manual)", 2020-04-10T07:07:45.270 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "leader_instance_id": "4415", 2020-04-10T07:07:45.271 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "leader": false, 2020-04-10T07:07:45.271 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "local_cluster_admin_socket": "/var/run/ceph/client.mirror.2.15704.cluster1.94209049498728.asok", 2020-04-10T07:07:45.271 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "remote_cluster_admin_socket": "/var/run/ceph/client.mirror.15704.cluster2-DNE.94209061786728.asok", 2020-04-10T07:07:45.271 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "sync_throttler": { 2020-04-10T07:07:45.271 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "max_parallel_requests": 5, 2020-04-10T07:07:45.271 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "running_requests": 0, 2020-04-10T07:07:45.271 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "waiting_requests": 0 2020-04-10T07:07:45.272 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: }, 2020-04-10T07:07:45.272 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "deletion_throttler": { 2020-04-10T07:07:45.272 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "max_parallel_requests": 1, 2020-04-10T07:07:45.272 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "running_requests": 0, 2020-04-10T07:07:45.272 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "waiting_requests": 0 2020-04-10T07:07:45.272 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: }, 2020-04-10T07:07:45.273 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "image_replayers": [], 2020-04-10T07:07:45.273 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "namespaces": [ 2020-04-10T07:07:45.273 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: { 2020-04-10T07:07:45.273 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "name": "ns1", 2020-04-10T07:07:45.273 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "image_replayers": [] 2020-04-10T07:07:45.273 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: }, 2020-04-10T07:07:45.273 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: { 2020-04-10T07:07:45.274 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "name": "ns2", 2020-04-10T07:07:45.274 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "image_replayers": [] 2020-04-10T07:07:45.274 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: } 2020-04-10T07:07:45.274 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: ] 2020-04-10T07:07:45.274 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: }, 2020-04-10T07:07:45.274 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: { 2020-04-10T07:07:45.274 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "peer": "uuid: 467a3104-15d9-4d31-90d5-8005aa7c9eeb cluster: cluster2-DNE client: client.mirror", 2020-04-10T07:07:45.275 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "pool": "mirror_parent", 2020-04-10T07:07:45.275 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "instance_id": "4443", 2020-04-10T07:07:45.275 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "state": "stopped (manual)", 2020-04-10T07:07:45.275 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "leader_instance_id": "4436", 2020-04-10T07:07:45.275 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "leader": false, 2020-04-10T07:07:45.275 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "local_cluster_admin_socket": "/var/run/ceph/client.mirror.2.15704.cluster1.94209074779240.asok", 2020-04-10T07:07:45.275 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "remote_cluster_admin_socket": "/var/run/ceph/client.mirror.15704.cluster2-DNE.94209087034472.asok", 2020-04-10T07:07:45.276 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "sync_throttler": { 2020-04-10T07:07:45.276 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "max_parallel_requests": 5, 2020-04-10T07:07:45.276 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "running_requests": 0, 2020-04-10T07:07:45.276 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "waiting_requests": 0 2020-04-10T07:07:45.276 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: }, 2020-04-10T07:07:45.276 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "deletion_throttler": { 2020-04-10T07:07:45.277 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "max_parallel_requests": 1, 2020-04-10T07:07:45.277 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "running_requests": 0, 2020-04-10T07:07:45.277 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "waiting_requests": 0 2020-04-10T07:07:45.277 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: }, 2020-04-10T07:07:45.277 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "image_replayers": [], 2020-04-10T07:07:45.277 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "namespaces": [] 2020-04-10T07:07:45.277 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: } 2020-04-10T07:07:45.278 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: ] 2020-04-10T07:07:45.278 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:} 2020-04-10T07:07:45.280 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:3 rbd mirror status 2020-04-10T07:07:45.280 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:3 2020-04-10T07:07:45.280 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:45.280 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:45.281 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:3 cluster instance 2020-04-10T07:07:45.281 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:3 2020-04-10T07:07:45.281 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:45.281 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:45.281 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:45.281 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:07:45.282 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 3 = cluster1:3 ] 2020-04-10T07:07:45.282 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:45.282 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:45.282 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=3 2020-04-10T07:07:45.282 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:07:45.282 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:3 cluster1 2020-04-10T07:07:45.282 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:3 2020-04-10T07:07:45.283 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:45.283 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:45.283 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:3 local_cluster instance 2020-04-10T07:07:45.283 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:3 2020-04-10T07:07:45.283 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:45.283 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:45.283 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:45.284 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:07:45.284 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 3 = cluster1:3 ] 2020-04-10T07:07:45.284 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:45.284 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:45.284 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=3 2020-04-10T07:07:45.284 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:07:45.285 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.3 admin socket 2020-04-10T07:07:45.297 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:07:45.297 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:07:45.298 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:07:45.298 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status 2020-04-10T07:07:45.384 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:{ 2020-04-10T07:07:45.384 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "pool_replayers": [ 2020-04-10T07:07:45.384 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: { 2020-04-10T07:07:45.385 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "peer": "uuid: 2571f49c-d767-4112-ae12-d8b71263d58b cluster: cluster2-DNE client: client.mirror", 2020-04-10T07:07:45.385 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "pool": "mirror", 2020-04-10T07:07:45.385 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "instance_id": "4433", 2020-04-10T07:07:45.385 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "state": "stopped (manual)", 2020-04-10T07:07:45.385 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "leader_instance_id": "4415", 2020-04-10T07:07:45.385 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "leader": false, 2020-04-10T07:07:45.385 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "local_cluster_admin_socket": "/var/run/ceph/client.mirror.3.15803.cluster1.94593015551080.asok", 2020-04-10T07:07:45.386 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "remote_cluster_admin_socket": "/var/run/ceph/client.mirror.15803.cluster2-DNE.94593028240488.asok", 2020-04-10T07:07:45.386 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "sync_throttler": { 2020-04-10T07:07:45.387 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "max_parallel_requests": 5, 2020-04-10T07:07:45.387 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "running_requests": 0, 2020-04-10T07:07:45.388 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "waiting_requests": 0 2020-04-10T07:07:45.388 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: }, 2020-04-10T07:07:45.388 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "deletion_throttler": { 2020-04-10T07:07:45.388 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "max_parallel_requests": 1, 2020-04-10T07:07:45.388 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "running_requests": 0, 2020-04-10T07:07:45.388 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "waiting_requests": 0 2020-04-10T07:07:45.389 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: }, 2020-04-10T07:07:45.389 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "image_replayers": [], 2020-04-10T07:07:45.389 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "namespaces": [ 2020-04-10T07:07:45.389 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: { 2020-04-10T07:07:45.389 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "name": "ns1", 2020-04-10T07:07:45.389 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "image_replayers": [] 2020-04-10T07:07:45.389 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: }, 2020-04-10T07:07:45.390 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: { 2020-04-10T07:07:45.390 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "name": "ns2", 2020-04-10T07:07:45.390 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "image_replayers": [] 2020-04-10T07:07:45.390 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: } 2020-04-10T07:07:45.390 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: ] 2020-04-10T07:07:45.390 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: }, 2020-04-10T07:07:45.390 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: { 2020-04-10T07:07:45.391 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "peer": "uuid: 467a3104-15d9-4d31-90d5-8005aa7c9eeb cluster: cluster2-DNE client: client.mirror", 2020-04-10T07:07:45.391 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "pool": "mirror_parent", 2020-04-10T07:07:45.391 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "instance_id": "4447", 2020-04-10T07:07:45.391 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "state": "stopped (manual)", 2020-04-10T07:07:45.391 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "leader_instance_id": "4436", 2020-04-10T07:07:45.391 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "leader": false, 2020-04-10T07:07:45.392 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "local_cluster_admin_socket": "/var/run/ceph/client.mirror.3.15803.cluster1.94593041282152.asok", 2020-04-10T07:07:45.392 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "remote_cluster_admin_socket": "/var/run/ceph/client.mirror.15803.cluster2-DNE.94593053561960.asok", 2020-04-10T07:07:45.392 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "sync_throttler": { 2020-04-10T07:07:45.392 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "max_parallel_requests": 5, 2020-04-10T07:07:45.392 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "running_requests": 0, 2020-04-10T07:07:45.392 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "waiting_requests": 0 2020-04-10T07:07:45.392 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: }, 2020-04-10T07:07:45.393 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "deletion_throttler": { 2020-04-10T07:07:45.393 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "max_parallel_requests": 1, 2020-04-10T07:07:45.393 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "running_requests": 0, 2020-04-10T07:07:45.393 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "waiting_requests": 0 2020-04-10T07:07:45.393 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: }, 2020-04-10T07:07:45.393 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "image_replayers": [], 2020-04-10T07:07:45.393 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "namespaces": [] 2020-04-10T07:07:45.394 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: } 2020-04-10T07:07:45.394 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: ] 2020-04-10T07:07:45.394 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:} 2020-04-10T07:07:45.396 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ testlog TEST: test image rename 2020-04-10T07:07:45.397 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ tee -a /tmp/tmp.8SwBCsYPSC/rbd-mirror.test.log 2020-04-10T07:07:45.397 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ date +%F %T 2020-04-10T07:07:45.397 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10 07:07:45 TEST: test image rename 2020-04-10T07:07:45.397 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:2020-04-10 07:07:45 TEST: test image rename 2020-04-10T07:07:45.397 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ new_name=test_RENAMED 2020-04-10T07:07:45.397 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rename_image cluster2 mirror test test_RENAMED 2020-04-10T07:07:45.398 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:07:45.398 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:45.398 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:45.398 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local new_name=test_RENAMED 2020-04-10T07:07:45.398 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster2 rename mirror/test mirror/test_RENAMED 2020-04-10T07:07:45.489 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mirror_image_snapshot cluster2 mirror test_RENAMED 2020-04-10T07:07:45.490 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:07:45.490 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:45.490 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_RENAMED 2020-04-10T07:07:45.490 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test_RENAMED 2020-04-10T07:07:45.933 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Snapshot ID: 12 2020-04-10T07:07:45.943 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_started cluster1 mirror test_RENAMED 2020-04-10T07:07:45.943 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:45.943 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:45.943 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_RENAMED 2020-04-10T07:07:45.943 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_state cluster1 mirror test_RENAMED started 2020-04-10T07:07:45.944 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:45.944 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:45.944 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_RENAMED 2020-04-10T07:07:45.944 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state=started 2020-04-10T07:07:45.944 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local s 2020-04-10T07:07:45.944 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:07:46.951 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_image_replay_state cluster1 mirror test_RENAMED started 2020-04-10T07:07:46.951 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:46.951 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:46.951 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_RENAMED 2020-04-10T07:07:46.951 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local test_state=started 2020-04-10T07:07:46.952 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_result 2020-04-10T07:07:46.952 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_state=stopped 2020-04-10T07:07:46.952 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test_RENAMED 2020-04-10T07:07:46.952 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_instance=cluster1 2020-04-10T07:07:46.952 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:46.952 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:46.953 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance=cluster1 2020-04-10T07:07:46.953 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local loop_instance 2020-04-10T07:07:46.953 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ + sleep 0 2020-04-10T07:07:46.953 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:grep -i state 2020-04-10T07:07:46.953 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ cluster1 != cluster1 ] 2020-04-10T07:07:46.953 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:07:46.953 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test_RENAMED 2020-04-10T07:07:46.954 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:07:46.954 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:46.954 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:46.954 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:07:46.954 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:46.954 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:46.954 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:46.955 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:46.955 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:46.955 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:46.955 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:46.955 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:46.955 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:46.955 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:46.955 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster1 2020-04-10T07:07:46.956 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:07:46.956 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:46.956 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:46.956 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:07:46.956 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:46.956 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:46.957 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:46.957 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:46.957 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:46.958 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:46.958 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:46.959 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:46.959 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:46.959 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:46.959 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:07:46.971 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:46.972 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:46.972 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:46.972 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test_RENAMED 2020-04-10T07:07:47.086 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:Invalid command: unused arguments: ['mirror/test_RENAMED'] 2020-04-10T07:07:47.086 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status : get status for rbd mirror 2020-04-10T07:07:47.086 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:07:47.088 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test_RENAMED 2020-04-10T07:07:47.089 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:1 2020-04-10T07:07:47.089 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:47.089 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:47.089 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 cluster instance 2020-04-10T07:07:47.089 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:07:47.089 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:47.089 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:47.090 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:47.090 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:47.090 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:07:47.090 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:47.090 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:47.090 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:07:47.090 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:47.091 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:1 cluster1 2020-04-10T07:07:47.091 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:1 2020-04-10T07:07:47.091 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:47.091 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:47.091 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2020-04-10T07:07:47.091 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:07:47.092 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:47.092 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:47.092 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:47.092 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:47.092 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:07:47.092 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:47.092 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:47.093 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:07:47.093 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:47.093 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2020-04-10T07:07:47.102 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:47.102 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:47.102 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:47.102 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test_RENAMED 2020-04-10T07:07:47.200 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:47.200 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ status_result= "state": "Replaying" 2020-04-10T07:07:47.200 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo "state": "Replaying" 2020-04-10T07:07:47.201 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep -i Replaying 2020-04-10T07:07:47.202 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "state": "Replaying" 2020-04-10T07:07:47.202 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ current_state=started 2020-04-10T07:07:47.202 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test started = started 2020-04-10T07:07:47.202 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:47.203 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test_RENAMED up+replaying 2020-04-10T07:07:47.203 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:47.203 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:47.203 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_RENAMED 2020-04-10T07:07:47.203 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:07:47.203 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:07:47.203 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:07:47.204 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:07:48.203 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_status_in_pool_dir cluster1 mirror test_RENAMED up+replaying 2020-04-10T07:07:48.203 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:48.203 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:48.203 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_RENAMED 2020-04-10T07:07:48.203 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:07:48.204 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:07:48.204 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:07:48.204 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror-test_RENAMED.mirror_status 2020-04-10T07:07:48.204 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror-test_RENAMED.mirror_status 2020-04-10T07:07:48.204 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:07:48.205 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_RENAMED.mirror_status 2020-04-10T07:07:48.205 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test_RENAMED 2020-04-10T07:07:48.205 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ tee /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_RENAMED.mirror_status 2020-04-10T07:07:48.283 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:test_RENAMED: 2020-04-10T07:07:48.283 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: global_id: 687e9f7d-ae46-4324-b8f7-7c16605e0fa6 2020-04-10T07:07:48.283 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+replaying 2020-04-10T07:07:48.284 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: replaying, {"replay_state": "idle", "remote_snapshot_timestamp": 1586502409, "local_snapshot_timestamp": 1586502409} 2020-04-10T07:07:48.284 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: service: mirror.1 on smithi204 2020-04-10T07:07:48.284 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:07:39 2020-04-10T07:07:48.291 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ state: .*up+replaying /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_RENAMED.mirror_status 2020-04-10T07:07:48.293 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+replaying 2020-04-10T07:07:48.294 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ description: .* /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_RENAMED.mirror_status 2020-04-10T07:07:48.295 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: replaying, {"replay_state": "idle", "remote_snapshot_timestamp": 1586502409, "local_snapshot_timestamp": 1586502409} 2020-04-10T07:07:48.296 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:07:48.296 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+replaying 2020-04-10T07:07:48.296 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^up+ 2020-04-10T07:07:48.297 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+replaying 2020-04-10T07:07:48.297 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep service: *mirror..* on /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_RENAMED.mirror_status 2020-04-10T07:07:48.299 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: service: mirror.1 on smithi204 2020-04-10T07:07:48.299 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_RENAMED.mirror_status 2020-04-10T07:07:48.302 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update=2020-04-10 07:07:39 2020-04-10T07:07:48.302 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test_RENAMED up+replaying 2020-04-10 07:07:39 2020-04-10T07:07:48.302 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:48.302 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:48.302 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_RENAMED 2020-04-10T07:07:48.302 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:07:48.303 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local prev_last_update=2020-04-10 07:07:39 2020-04-10T07:07:48.303 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror.mirror_status 2020-04-10T07:07:48.303 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror.mirror_status 2020-04-10T07:07:48.304 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:07:48.304 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror.mirror_status 2020-04-10T07:07:48.304 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2020-04-10T07:07:48.349 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: failed to query services: (13) Permission denied 2020-04-10T07:07:48.399 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update state 2020-04-10T07:07:48.400 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test_RENAMED']/last_update 2020-04-10T07:07:48.404 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ last_update=2020-04-10 07:07:39 2020-04-10T07:07:48.405 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test_RENAMED']/state 2020-04-10T07:07:48.410 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ state=up+replaying 2020-04-10T07:07:48.410 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+replaying 2020-04-10T07:07:48.410 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep up+replaying 2020-04-10T07:07:48.412 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+replaying 2020-04-10T07:07:48.412 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:48.412 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:48.412 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test_RENAMED 2020-04-10T07:07:48.413 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_instance=cluster1 2020-04-10T07:07:48.413 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:48.413 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:48.413 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance=cluster1 2020-04-10T07:07:48.413 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local loop_instance 2020-04-10T07:07:48.413 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0 2020-04-10T07:07:48.416 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ cluster1 != cluster1 ] 2020-04-10T07:07:48.416 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:07:48.416 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test_RENAMED 2020-04-10T07:07:48.416 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:07:48.417 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:48.417 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:48.417 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:07:48.417 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:48.417 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:48.417 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:48.417 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:48.418 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:48.418 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:48.418 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:48.418 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:48.418 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:48.418 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:48.418 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster1 2020-04-10T07:07:48.419 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:07:48.419 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:48.419 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:48.419 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:07:48.419 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:48.419 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:48.419 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:48.419 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:48.420 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:48.420 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:48.420 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:48.420 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:48.420 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:48.420 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:48.420 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:07:48.434 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:48.434 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:48.435 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:48.435 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test_RENAMED 2020-04-10T07:07:48.527 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:Invalid command: unused arguments: ['mirror/test_RENAMED'] 2020-04-10T07:07:48.527 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status : get status for rbd mirror 2020-04-10T07:07:48.527 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:07:48.529 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test_RENAMED 2020-04-10T07:07:48.530 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:1 2020-04-10T07:07:48.530 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:48.530 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:48.530 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 cluster instance 2020-04-10T07:07:48.530 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:07:48.530 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:48.530 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:48.531 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:48.531 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:48.531 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:07:48.531 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:48.531 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:48.531 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:07:48.531 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:48.532 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:1 cluster1 2020-04-10T07:07:48.532 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:1 2020-04-10T07:07:48.532 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:48.532 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:48.532 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2020-04-10T07:07:48.532 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:07:48.532 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:48.532 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:48.533 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:48.533 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:48.533 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:07:48.533 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:48.533 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:48.533 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:07:48.533 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:48.534 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2020-04-10T07:07:48.551 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:48.551 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:48.551 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:48.551 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test_RENAMED 2020-04-10T07:07:48.639 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:{ 2020-04-10T07:07:48.640 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "name": "mirror/test_RENAMED", 2020-04-10T07:07:48.640 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "state": "Replaying" 2020-04-10T07:07:48.640 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:} 2020-04-10T07:07:48.647 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:48.647 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemons cluster1 rbd mirror restart mirror/test_RENAMED 2020-04-10T07:07:48.647 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_instance=cluster1 2020-04-10T07:07:48.648 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:48.648 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:48.648 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance=cluster1 2020-04-10T07:07:48.648 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local loop_instance 2020-04-10T07:07:48.648 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0 2020-04-10T07:07:48.649 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ cluster1 != cluster1 ] 2020-04-10T07:07:48.649 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:07:48.651 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:0 rbd mirror restart mirror/test_RENAMED 2020-04-10T07:07:48.651 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:07:48.652 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:48.652 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:48.652 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:07:48.652 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:48.652 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:48.652 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:48.653 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:48.653 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:48.653 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:48.653 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:48.653 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:48.653 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:48.654 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:48.654 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster1 2020-04-10T07:07:48.654 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:07:48.654 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:48.654 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:48.655 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:07:48.655 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:48.655 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:48.655 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:48.655 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:48.655 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:48.655 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:48.656 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:48.656 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:48.656 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:48.656 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:48.656 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:07:48.674 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:48.674 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:48.674 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:48.675 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror restart mirror/test_RENAMED 2020-04-10T07:07:48.765 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:Invalid command: unused arguments: ['mirror/test_RENAMED'] 2020-04-10T07:07:48.765 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror restart : restart rbd mirror 2020-04-10T07:07:48.765 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:07:48.767 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:1 rbd mirror restart mirror/test_RENAMED 2020-04-10T07:07:48.767 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:1 2020-04-10T07:07:48.767 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:48.767 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:48.767 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 cluster instance 2020-04-10T07:07:48.767 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:07:48.768 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:48.768 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:48.768 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:48.768 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:48.768 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:07:48.768 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:48.768 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:48.769 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:07:48.769 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:48.769 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:1 cluster1 2020-04-10T07:07:48.769 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:1 2020-04-10T07:07:48.769 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:48.769 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:48.769 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2020-04-10T07:07:48.770 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:07:48.770 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:48.770 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:48.770 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:48.770 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:48.770 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:07:48.770 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:48.770 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:48.771 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:07:48.771 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:48.771 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2020-04-10T07:07:48.787 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:48.788 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:48.788 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:48.788 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror restart mirror/test_RENAMED 2020-04-10T07:07:48.886 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:48.886 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_started cluster1 mirror test_RENAMED 2020-04-10T07:07:48.886 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:48.887 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:48.887 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_RENAMED 2020-04-10T07:07:48.887 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_state cluster1 mirror test_RENAMED started 2020-04-10T07:07:48.887 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:48.887 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:48.887 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_RENAMED 2020-04-10T07:07:48.887 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state=started 2020-04-10T07:07:48.888 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local s 2020-04-10T07:07:48.888 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:07:49.887 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_image_replay_state cluster1 mirror test_RENAMED started 2020-04-10T07:07:49.887 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:49.887 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:49.887 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_RENAMED 2020-04-10T07:07:49.888 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local test_state=started 2020-04-10T07:07:49.888 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_result 2020-04-10T07:07:49.888 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_state=stopped 2020-04-10T07:07:49.889 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test_RENAMED+ 2020-04-10T07:07:49.889 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:grep -i+ statelocal 2020-04-10T07:07:49.889 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: cluster_instance=cluster1 2020-04-10T07:07:49.889 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:49.889 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:49.890 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance=cluster1 2020-04-10T07:07:49.890 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local loop_instance 2020-04-10T07:07:49.890 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0 2020-04-10T07:07:49.890 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ cluster1 != cluster1 ] 2020-04-10T07:07:49.892 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:07:49.893 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test_RENAMED 2020-04-10T07:07:49.893 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:07:49.893 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:49.893 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:49.893 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:07:49.893 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:49.893 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:49.894 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:49.894 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:49.894 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:49.894 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:49.894 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:49.894 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:49.894 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:49.895 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:49.895 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster1 2020-04-10T07:07:49.895 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:07:49.895 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:49.895 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:49.895 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:07:49.895 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:49.896 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:49.896 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:49.896 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:49.896 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:49.896 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:49.896 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:49.896 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:49.897 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:49.897 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:49.897 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:07:49.911 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:49.912 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:49.912 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:49.912 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test_RENAMED 2020-04-10T07:07:50.006 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:Invalid command: unused arguments: ['mirror/test_RENAMED'] 2020-04-10T07:07:50.006 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status : get status for rbd mirror 2020-04-10T07:07:50.006 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:07:50.011 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test_RENAMED 2020-04-10T07:07:50.011 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:1 2020-04-10T07:07:50.011 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:50.011 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:50.011 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 cluster instance 2020-04-10T07:07:50.013 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:07:50.013 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:50.013 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:50.013 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:50.013 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:50.014 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:07:50.014 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:50.014 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:50.014 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:07:50.014 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:50.014 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:1 cluster1 2020-04-10T07:07:50.014 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:1 2020-04-10T07:07:50.015 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:50.015 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:50.015 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2020-04-10T07:07:50.015 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:07:50.015 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:50.015 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:50.015 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:50.015 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:50.016 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:07:50.016 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:50.016 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:50.016 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:07:50.016 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:50.016 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2020-04-10T07:07:50.028 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:50.028 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:50.028 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:50.028 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test_RENAMED 2020-04-10T07:07:50.126 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:50.127 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ status_result= "state": "Replaying" 2020-04-10T07:07:50.127 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo "state": "Replaying" 2020-04-10T07:07:50.127 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep -i Replaying 2020-04-10T07:07:50.129 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "state": "Replaying" 2020-04-10T07:07:50.129 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ current_state=started 2020-04-10T07:07:50.129 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test started = started 2020-04-10T07:07:50.129 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:50.129 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test_RENAMED up+replaying 2020-04-10T07:07:50.129 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:50.130 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:50.130 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_RENAMED 2020-04-10T07:07:50.130 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:07:50.130 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:07:50.130 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:07:50.130 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:07:51.130 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_status_in_pool_dir cluster1 mirror test_RENAMED up+replaying 2020-04-10T07:07:51.131 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:51.131 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:51.131 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_RENAMED 2020-04-10T07:07:51.131 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:07:51.131 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:07:51.131 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:07:51.131 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror-test_RENAMED.mirror_status 2020-04-10T07:07:51.132 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror-test_RENAMED.mirror_status 2020-04-10T07:07:51.132 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:07:51.132 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_RENAMED.mirror_status 2020-04-10T07:07:51.133 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test_RENAMED 2020-04-10T07:07:51.133 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ tee /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_RENAMED.mirror_status 2020-04-10T07:07:51.203 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:test_RENAMED: 2020-04-10T07:07:51.203 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: global_id: 687e9f7d-ae46-4324-b8f7-7c16605e0fa6 2020-04-10T07:07:51.203 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+replaying 2020-04-10T07:07:51.203 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: replaying, {"replay_state": "idle", "remote_snapshot_timestamp": 1586502465, "local_snapshot_timestamp": 1586502465} 2020-04-10T07:07:51.204 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: service: mirror.1 on smithi204 2020-04-10T07:07:51.204 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:07:48 2020-04-10T07:07:51.211 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ state: .*up+replaying /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_RENAMED.mirror_status 2020-04-10T07:07:51.213 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+replaying 2020-04-10T07:07:51.213 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ description: .* /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_RENAMED.mirror_status 2020-04-10T07:07:51.215 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: replaying, {"replay_state": "idle", "remote_snapshot_timestamp": 1586502465, "local_snapshot_timestamp": 1586502465} 2020-04-10T07:07:51.216 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:07:51.216 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+replaying 2020-04-10T07:07:51.216 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^up+ 2020-04-10T07:07:51.217 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+replaying 2020-04-10T07:07:51.217 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep service: *mirror..* on /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_RENAMED.mirror_status 2020-04-10T07:07:51.218 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: service: mirror.1 on smithi204 2020-04-10T07:07:51.218 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_RENAMED.mirror_status 2020-04-10T07:07:51.221 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update=2020-04-10 07:07:48 2020-04-10T07:07:51.221 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test_RENAMED up+replaying 2020-04-10 07:07:48 2020-04-10T07:07:51.222 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:51.222 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:51.222 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_RENAMED 2020-04-10T07:07:51.222 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:07:51.222 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local prev_last_update=2020-04-10 07:07:48 2020-04-10T07:07:51.222 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror.mirror_status 2020-04-10T07:07:51.222 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror.mirror_status 2020-04-10T07:07:51.223 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:07:51.225 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror.mirror_status 2020-04-10T07:07:51.225 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2020-04-10T07:07:51.273 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: failed to query services: (13) Permission denied 2020-04-10T07:07:51.322 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update state 2020-04-10T07:07:51.322 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test_RENAMED']/last_update 2020-04-10T07:07:51.328 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ last_update=2020-04-10 07:07:48 2020-04-10T07:07:51.328 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test_RENAMED']/state 2020-04-10T07:07:51.333 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ state=up+replaying 2020-04-10T07:07:51.333 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+replaying 2020-04-10T07:07:51.334 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep up+replaying 2020-04-10T07:07:51.335 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+replaying 2020-04-10T07:07:51.335 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:51.335 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:51.336 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rename_image cluster2 mirror test_RENAMED test 2020-04-10T07:07:51.336 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:07:51.336 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:51.336 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_RENAMED 2020-04-10T07:07:51.336 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local new_name=test 2020-04-10T07:07:51.336 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster2 rename mirror/test_RENAMED mirror/test 2020-04-10T07:07:51.635 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mirror_image_snapshot cluster2 mirror test 2020-04-10T07:07:51.635 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:07:51.635 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:51.636 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:51.636 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test 2020-04-10T07:07:52.101 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Snapshot ID: 14 2020-04-10T07:07:52.110 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_started cluster1 mirror test 2020-04-10T07:07:52.110 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:52.111 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:52.111 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:52.111 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2020-04-10T07:07:52.111 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:52.111 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:52.111 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:52.111 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state=started 2020-04-10T07:07:52.112 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local s 2020-04-10T07:07:52.112 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:07:53.112 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_image_replay_state cluster1 mirror test started 2020-04-10T07:07:53.112 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:53.112 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:53.112 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:53.113 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local test_state=started 2020-04-10T07:07:53.113 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_result 2020-04-10T07:07:53.113 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_state=stopped 2020-04-10T07:07:53.113 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2020-04-10T07:07:53.113 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ + local cluster_instance=cluster1 2020-04-10T07:07:53.113 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:grep+ -i stateshift 2020-04-10T07:07:53.113 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: 2020-04-10T07:07:53.114 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:53.114 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance=cluster1 2020-04-10T07:07:53.114 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local loop_instance 2020-04-10T07:07:53.114 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0 2020-04-10T07:07:53.116 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ cluster1 != cluster1 ] 2020-04-10T07:07:53.117 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:07:53.117 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2020-04-10T07:07:53.117 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:07:53.117 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:53.117 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:53.117 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:07:53.117 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:53.117 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:53.118 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:53.118 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:53.118 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:53.118 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:53.118 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:53.118 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:53.118 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:53.119 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:53.119 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster1 2020-04-10T07:07:53.119 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:07:53.119 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:53.119 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:53.119 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:07:53.120 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:53.120 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:53.120 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:53.120 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:53.120 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:53.120 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:53.120 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:53.121 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:53.121 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:53.121 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:53.121 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:07:53.133 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:53.133 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:53.133 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:53.133 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2020-04-10T07:07:53.218 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:no valid command found; 1 closest matches: 2020-04-10T07:07:53.218 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status mirror/test1 2020-04-10T07:07:53.218 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:07:53.220 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2020-04-10T07:07:53.220 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:1 2020-04-10T07:07:53.220 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:53.220 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:53.221 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 cluster instance 2020-04-10T07:07:53.221 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:07:53.221 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:53.221 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:53.221 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:53.221 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:53.221 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:07:53.222 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:53.222 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:53.222 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:07:53.222 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:53.222 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:1 cluster1 2020-04-10T07:07:53.222 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:1 2020-04-10T07:07:53.222 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:53.222 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:53.223 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2020-04-10T07:07:53.223 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:07:53.223 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:53.223 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:53.223 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:53.223 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:53.223 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:07:53.224 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:53.224 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:53.224 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:07:53.224 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:53.224 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2020-04-10T07:07:53.240 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:53.240 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:53.240 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:53.240 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2020-04-10T07:07:53.326 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:no valid command found; 1 closest matches: 2020-04-10T07:07:53.326 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status mirror/test_RENAMED 2020-04-10T07:07:53.327 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:07:53.331 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:2 rbd mirror status mirror/test 2020-04-10T07:07:53.331 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:2 2020-04-10T07:07:53.331 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:53.331 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:53.331 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 cluster instance 2020-04-10T07:07:53.331 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:07:53.332 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:53.332 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:53.332 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:53.332 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:07:53.332 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:07:53.332 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:53.332 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:53.333 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:07:53.333 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:07:53.333 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:2 cluster1 2020-04-10T07:07:53.333 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:2 2020-04-10T07:07:53.333 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:53.333 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:53.333 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2020-04-10T07:07:53.333 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:07:53.334 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:53.334 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:53.334 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:53.334 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:07:53.334 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:07:53.334 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:53.334 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:53.335 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:07:53.335 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:07:53.335 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2020-04-10T07:07:53.349 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:07:53.349 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:07:53.350 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:07:53.350 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2020-04-10T07:07:53.435 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:Invalid command: unused arguments: ['mirror/test'] 2020-04-10T07:07:53.436 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status : get status for rbd mirror 2020-04-10T07:07:53.436 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:07:53.438 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:3 rbd mirror status mirror/test 2020-04-10T07:07:53.438 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:3 2020-04-10T07:07:53.438 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:53.439 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:53.439 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:3 cluster instance 2020-04-10T07:07:53.439 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:3 2020-04-10T07:07:53.439 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:53.439 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:53.439 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:53.440 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:07:53.442 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 3 = cluster1:3 ] 2020-04-10T07:07:53.442 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:53.442 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:53.442 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=3 2020-04-10T07:07:53.442 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:07:53.442 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:3 cluster1 2020-04-10T07:07:53.443 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:3 2020-04-10T07:07:53.443 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:53.443 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:53.443 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:3 local_cluster instance 2020-04-10T07:07:53.443 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:3 2020-04-10T07:07:53.443 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:53.444 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:53.444 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:53.444 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:07:53.444 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 3 = cluster1:3 ] 2020-04-10T07:07:53.444 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:53.445 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:53.445 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=3 2020-04-10T07:07:53.445 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:07:53.445 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.3 admin socket 2020-04-10T07:07:53.461 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:07:53.462 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:07:53.462 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:07:53.462 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/test 2020-04-10T07:07:53.545 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:Invalid command: unused arguments: ['mirror/test'] 2020-04-10T07:07:53.546 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status : get status for rbd mirror 2020-04-10T07:07:53.546 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:07:53.547 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:07:54.548 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ cluster1 != cluster1 ] 2020-04-10T07:07:54.548 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:07:54.550 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2020-04-10T07:07:54.551 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:07:54.551 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:54.551 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:54.551 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:07:54.551 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:54.551 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:54.552 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:54.552 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:54.552 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:54.552 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:54.552 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:54.552 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:54.553 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:54.553 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:54.553 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster1 2020-04-10T07:07:54.553 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:07:54.553 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:54.553 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:54.554 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:07:54.554 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:07:54.554 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:54.554 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:54.554 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:54.554 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:54.554 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:07:54.555 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:54.555 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:54.555 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:07:54.555 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:07:54.555 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:07:54.572 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:54.573 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:54.573 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:07:54.573 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2020-04-10T07:07:54.671 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:no valid command found; 1 closest matches: 2020-04-10T07:07:54.672 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status mirror/test1 2020-04-10T07:07:54.672 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:07:54.672 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2020-04-10T07:07:54.672 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:1 2020-04-10T07:07:54.672 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:07:54.672 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:54.672 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 cluster instance 2020-04-10T07:07:54.673 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:07:54.673 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:07:54.673 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:54.673 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:54.673 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:54.673 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:07:54.673 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:07:54.674 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:54.674 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:07:54.674 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:54.674 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:1 cluster1 2020-04-10T07:07:54.674 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:1 2020-04-10T07:07:54.674 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:54.674 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:07:54.675 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2020-04-10T07:07:54.675 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:07:54.675 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:07:54.675 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:07:54.675 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:07:54.675 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:54.675 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:07:54.675 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:07:54.676 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:07:54.676 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:07:54.676 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:07:54.676 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2020-04-10T07:07:54.692 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:54.692 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:54.692 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:07:54.693 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2020-04-10T07:07:54.795 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:54.796 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ status_result= "state": "Replaying" 2020-04-10T07:07:54.796 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo "state": "Replaying" 2020-04-10T07:07:54.796 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep -i Replaying 2020-04-10T07:07:54.798 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "state": "Replaying" 2020-04-10T07:07:54.798 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ current_state=started 2020-04-10T07:07:54.798 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test started = started 2020-04-10T07:07:54.798 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:07:54.798 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ testlog TEST: test trash move restore 2020-04-10T07:07:54.798 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ tee -a /tmp/tmp.8SwBCsYPSC/rbd-mirror.test.log 2020-04-10T07:07:54.799 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ date +%F %T 2020-04-10T07:07:54.800 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10 07:07:54 TEST: test trash move restore 2020-04-10T07:07:54.800 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:2020-04-10 07:07:54 TEST: test trash move restore 2020-04-10T07:07:54.800 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_image_id cluster2 mirror test 2020-04-10T07:07:54.800 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:07:54.800 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:54.801 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:54.801 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster2 info mirror/test 2020-04-10T07:07:54.801 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -ne s/^.*block_name_prefix: rbd_data\.//p 2020-04-10T07:07:54.849 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ image_id=10e54538c85b 2020-04-10T07:07:54.849 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ trash_move cluster2 mirror test 2020-04-10T07:07:54.849 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:07:54.849 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:54.850 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:54.850 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster2 trash move mirror/test 2020-04-10T07:07:55.157 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_present cluster1 mirror test deleted 2020-04-10T07:07:55.157 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:55.158 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:55.158 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:55.158 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state=deleted 2020-04-10T07:07:55.158 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image_id= 2020-04-10T07:07:55.158 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local s 2020-04-10T07:07:55.158 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 2020-04-10T07:07:55.159 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_image_id cluster1 mirror test 2020-04-10T07:07:55.159 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:55.159 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:55.159 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:55.159 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster1 info mirror/test 2020-04-10T07:07:55.159 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -ne s/^.*block_name_prefix: rbd_data\.//p 2020-04-10T07:07:55.213 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ image_id=10f023e7a33d 2020-04-10T07:07:55.213 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0.1 2020-04-10T07:07:55.315 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_image_present cluster1 mirror test deleted 10f023e7a33d 2020-04-10T07:07:55.315 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:55.316 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:55.316 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:55.316 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local test_state=deleted 2020-04-10T07:07:55.316 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image_id=10f023e7a33d 2020-04-10T07:07:55.316 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_state=deleted 2020-04-10T07:07:55.316 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_image_id 2020-04-10T07:07:55.316 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_image_id cluster1 mirror test 2020-04-10T07:07:55.317 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:55.317 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:55.317 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:55.317 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster1 info mirror/test 2020-04-10T07:07:55.317 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -ne s/^.*block_name_prefix: rbd_data\.//p 2020-04-10T07:07:55.379 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ current_image_id=10f023e7a33d 2020-04-10T07:07:55.380 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 10f023e7a33d 2020-04-10T07:07:55.380 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -z 10f023e7a33d -o 10f023e7a33d = 10f023e7a33d 2020-04-10T07:07:55.380 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ current_state=present 2020-04-10T07:07:55.380 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test deleted = present 2020-04-10T07:07:55.380 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:07:56.382 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_image_present cluster1 mirror test deleted 10f023e7a33d 2020-04-10T07:07:56.382 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:56.382 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:56.382 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:56.383 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local test_state=deleted 2020-04-10T07:07:56.383 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image_id=10f023e7a33d 2020-04-10T07:07:56.383 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_state=deleted 2020-04-10T07:07:56.383 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_image_id 2020-04-10T07:07:56.383 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_image_id cluster1 mirror test 2020-04-10T07:07:56.383 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:56.383 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:56.383 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:56.384 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster1 info mirror/test 2020-04-10T07:07:56.384 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -ne s/^.*block_name_prefix: rbd_data\.//p 2020-04-10T07:07:56.455 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ current_image_id=10f023e7a33d 2020-04-10T07:07:56.455 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 10f023e7a33d 2020-04-10T07:07:56.455 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -z 10f023e7a33d -o 10f023e7a33d = 10f023e7a33d 2020-04-10T07:07:56.455 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ current_state=present 2020-04-10T07:07:56.455 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test deleted = present 2020-04-10T07:07:56.455 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 2 2020-04-10T07:07:58.461 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_image_present cluster1 mirror test deleted 10f023e7a33d 2020-04-10T07:07:58.461 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:58.461 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:58.461 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:58.461 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local test_state=deleted 2020-04-10T07:07:58.461 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image_id=10f023e7a33d 2020-04-10T07:07:58.462 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_state=deleted 2020-04-10T07:07:58.462 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_image_id 2020-04-10T07:07:58.462 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_image_id cluster1 mirror test 2020-04-10T07:07:58.462 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:07:58.462 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:07:58.462 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:07:58.462 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster1 info mirror/test 2020-04-10T07:07:58.463 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -ne s/^.*block_name_prefix: rbd_data\.//p 2020-04-10T07:07:58.527 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ current_image_id=10f023e7a33d 2020-04-10T07:07:58.527 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 10f023e7a33d 2020-04-10T07:07:58.528 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -z 10f023e7a33d -o 10f023e7a33d = 10f023e7a33d 2020-04-10T07:07:58.528 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ current_state=present 2020-04-10T07:07:58.528 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test deleted = present 2020-04-10T07:07:58.528 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 4 2020-04-10T07:08:02.530 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_image_present cluster1 mirror test deleted 10f023e7a33d 2020-04-10T07:08:02.530 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:02.530 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:02.530 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:02.531 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local test_state=deleted 2020-04-10T07:08:02.531 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image_id=10f023e7a33d 2020-04-10T07:08:02.531 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_state=deleted 2020-04-10T07:08:02.531 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_image_id 2020-04-10T07:08:02.531 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_image_id cluster1 mirror test 2020-04-10T07:08:02.531 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:02.531 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:02.532 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:02.532 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster1 info mirror/test 2020-04-10T07:08:02.532 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -ne s/^.*block_name_prefix: rbd_data\.//p 2020-04-10T07:08:02.569 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: error opening image test: (2) No such file or directory 2020-04-10T07:08:02.573 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ current_image_id= 2020-04-10T07:08:02.573 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 2020-04-10T07:08:02.574 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test deleted = deleted 2020-04-10T07:08:02.574 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:08:02.574 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ trash_restore cluster2 mirror 10e54538c85b 2020-04-10T07:08:02.574 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:08:02.574 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:02.574 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image_id=10e54538c85b 2020-04-10T07:08:02.575 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster2 trash restore mirror/10e54538c85b 2020-04-10T07:08:02.621 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ enable_mirror cluster2 mirror test snapshot 2020-04-10T07:08:02.622 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:08:02.622 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:02.623 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:02.623 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local mode=snapshot 2020-04-10T07:08:02.623 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/test snapshot 2020-04-10T07:08:03.135 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Mirroring enabled 2020-04-10T07:08:03.144 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_started cluster1 mirror test 2020-04-10T07:08:03.145 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:03.145 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:03.145 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:03.145 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2020-04-10T07:08:03.145 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:03.145 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:03.146 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:03.146 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state=started 2020-04-10T07:08:03.146 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local s 2020-04-10T07:08:03.146 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:08:04.145 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_image_replay_state cluster1 mirror test started 2020-04-10T07:08:04.146 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:04.146 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:04.146 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:04.146 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local test_state=started 2020-04-10T07:08:04.146 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_result 2020-04-10T07:08:04.146 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_state=stopped 2020-04-10T07:08:04.147 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemons+ cluster1 rbd mirror status mirror/test 2020-04-10T07:08:04.147 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:grep -i state 2020-04-10T07:08:04.148 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_instance=cluster1 2020-04-10T07:08:04.148 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:08:04.148 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:04.148 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance=cluster1 2020-04-10T07:08:04.148 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local loop_instance 2020-04-10T07:08:04.148 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0 2020-04-10T07:08:04.152 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ cluster1 != cluster1 ] 2020-04-10T07:08:04.152 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:08:04.152 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2020-04-10T07:08:04.152 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:08:04.152 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:08:04.153 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:04.153 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:08:04.153 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:08:04.153 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:08:04.154 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:04.154 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:04.154 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:08:04.154 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:08:04.154 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:08:04.154 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:04.155 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:08:04.155 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:08:04.155 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster1 2020-04-10T07:08:04.155 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:08:04.155 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:04.155 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:04.155 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:08:04.156 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:08:04.156 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:08:04.156 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:04.156 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:04.156 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:08:04.156 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:08:04.156 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:08:04.157 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:08:04.157 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:08:04.157 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:08:04.157 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:08:04.169 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:08:04.169 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:08:04.169 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:08:04.169 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2020-04-10T07:08:04.254 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:no valid command found; 1 closest matches: 2020-04-10T07:08:04.254 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status mirror/test1 2020-04-10T07:08:04.254 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:08:04.257 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2020-04-10T07:08:04.257 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:1 2020-04-10T07:08:04.257 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:08:04.257 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:04.258 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 cluster instance 2020-04-10T07:08:04.258 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:08:04.258 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:08:04.258 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:04.258 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:04.258 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:08:04.258 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:08:04.259 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:08:04.259 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:04.259 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:08:04.259 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:08:04.259 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:1 cluster1 2020-04-10T07:08:04.259 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:1 2020-04-10T07:08:04.259 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:04.260 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:04.260 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2020-04-10T07:08:04.260 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:08:04.260 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:08:04.260 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:04.260 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:04.260 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:08:04.261 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:08:04.261 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:08:04.261 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:08:04.261 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:08:04.261 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:08:04.261 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2020-04-10T07:08:04.278 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:08:04.279 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:08:04.279 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:08:04.279 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2020-04-10T07:08:04.364 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:Invalid command: unused arguments: ['mirror/test'] 2020-04-10T07:08:04.364 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status : get status for rbd mirror 2020-04-10T07:08:04.364 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:08:04.367 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:2 rbd mirror status mirror/test 2020-04-10T07:08:04.367 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:2 2020-04-10T07:08:04.367 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:08:04.367 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:04.367 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 cluster instance 2020-04-10T07:08:04.368 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:08:04.368 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:08:04.368 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:04.368 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:04.368 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:08:04.368 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:08:04.368 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:08:04.369 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:04.369 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:08:04.369 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:08:04.369 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:2 cluster1 2020-04-10T07:08:04.369 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:2 2020-04-10T07:08:04.369 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:04.370 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:04.370 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2020-04-10T07:08:04.370 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:08:04.370 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:08:04.370 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:04.370 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:04.370 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:08:04.370 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:08:04.371 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:08:04.371 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:08:04.371 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:08:04.371 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:08:04.372 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2020-04-10T07:08:04.388 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:08:04.388 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:08:04.388 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:08:04.389 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2020-04-10T07:08:04.473 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:Invalid command: unused arguments: ['mirror/test'] 2020-04-10T07:08:04.473 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status : get status for rbd mirror 2020-04-10T07:08:04.474 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:08:04.478 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:3 rbd mirror status mirror/test 2020-04-10T07:08:04.479 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:3 2020-04-10T07:08:04.479 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:08:04.479 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:04.479 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:3 cluster instance 2020-04-10T07:08:04.479 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:3 2020-04-10T07:08:04.479 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:08:04.479 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:04.480 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:04.480 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:08:04.480 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 3 = cluster1:3 ] 2020-04-10T07:08:04.480 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:08:04.480 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:04.480 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=3 2020-04-10T07:08:04.480 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:08:04.481 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:3 cluster1 2020-04-10T07:08:04.481 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:3 2020-04-10T07:08:04.481 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:04.481 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:04.481 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:3 local_cluster instance 2020-04-10T07:08:04.481 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:3 2020-04-10T07:08:04.481 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:08:04.482 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:04.482 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:04.482 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:08:04.482 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 3 = cluster1:3 ] 2020-04-10T07:08:04.482 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:08:04.482 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:08:04.482 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=3 2020-04-10T07:08:04.482 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:08:04.483 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.3 admin socket 2020-04-10T07:08:04.489 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:08:04.490 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:08:04.490 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:08:04.490 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/test 2020-04-10T07:08:04.581 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:Invalid command: unused arguments: ['mirror/test'] 2020-04-10T07:08:04.581 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status : get status for rbd mirror 2020-04-10T07:08:04.581 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:08:04.582 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:08:05.583 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ cluster1 != cluster1 ] 2020-04-10T07:08:05.584 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:08:05.586 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2020-04-10T07:08:05.586 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:08:05.587 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:08:05.587 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:05.587 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:08:05.587 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:08:05.587 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:08:05.587 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:05.587 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:05.588 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:08:05.588 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:08:05.588 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:08:05.588 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:05.588 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:08:05.588 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:08:05.588 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster1 2020-04-10T07:08:05.588 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:08:05.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:05.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:05.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:08:05.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:08:05.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:08:05.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:05.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:05.590 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:08:05.590 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:08:05.590 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:08:05.590 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:08:05.590 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:08:05.590 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:08:05.590 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:08:05.608 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:08:05.608 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:08:05.608 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:08:05.609 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2020-04-10T07:08:05.705 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:no valid command found; 1 closest matches: 2020-04-10T07:08:05.705 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status mirror/test1 2020-04-10T07:08:05.705 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:08:05.707 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2020-04-10T07:08:05.707 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:1 2020-04-10T07:08:05.707 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:08:05.708 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:05.708 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 cluster instance 2020-04-10T07:08:05.708 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:08:05.708 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:08:05.708 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:05.708 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:05.708 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:08:05.709 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:08:05.709 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:08:05.709 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:05.709 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:08:05.709 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:08:05.709 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:1 cluster1 2020-04-10T07:08:05.709 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:1 2020-04-10T07:08:05.710 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:05.710 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:05.710 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2020-04-10T07:08:05.710 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:08:05.710 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:08:05.710 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:05.710 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:05.710 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:08:05.711 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:08:05.711 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:08:05.711 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:08:05.711 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:08:05.711 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:08:05.711 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2020-04-10T07:08:05.727 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:08:05.727 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:08:05.727 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:08:05.727 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2020-04-10T07:08:05.816 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:Invalid command: unused arguments: ['mirror/test'] 2020-04-10T07:08:05.816 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status : get status for rbd mirror 2020-04-10T07:08:05.816 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:08:05.818 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:2 rbd mirror status mirror/test 2020-04-10T07:08:05.818 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:2 2020-04-10T07:08:05.819 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:08:05.819 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:05.819 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 cluster instance 2020-04-10T07:08:05.819 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:08:05.819 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:08:05.819 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:05.819 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:05.820 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:08:05.820 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:08:05.820 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:08:05.820 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:05.820 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:08:05.820 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:08:05.820 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:2 cluster1 2020-04-10T07:08:05.821 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:2 2020-04-10T07:08:05.821 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:05.821 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:05.821 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2020-04-10T07:08:05.821 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:08:05.821 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:08:05.821 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:05.822 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:05.822 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:08:05.822 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:08:05.822 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:08:05.822 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:08:05.822 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:08:05.822 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:08:05.822 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2020-04-10T07:08:05.839 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:08:05.839 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:08:05.839 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:08:05.839 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2020-04-10T07:08:05.936 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:08:05.936 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ status_result= "state": "Replaying" 2020-04-10T07:08:05.937 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo "state": "Replaying" 2020-04-10T07:08:05.937 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep -i Replaying 2020-04-10T07:08:05.938 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "state": "Replaying" 2020-04-10T07:08:05.939 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ current_state=started 2020-04-10T07:08:05.939 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test started = started 2020-04-10T07:08:05.939 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:08:05.939 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ testlog TEST: failover and failback 2020-04-10T07:08:05.939 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ tee -a /tmp/tmp.8SwBCsYPSC/rbd-mirror.test.log 2020-04-10T07:08:05.939 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ date +%F %T 2020-04-10T07:08:05.940 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10 07:08:05 TEST: failover and failback 2020-04-10T07:08:05.940 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:2020-04-10 07:08:05 TEST: failover and failback 2020-04-10T07:08:05.941 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ start_mirrors cluster2 2020-04-10T07:08:05.941 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:08:05.941 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:08:05.942 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ start_mirror cluster2:0 2020-04-10T07:08:05.942 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2:0 2020-04-10T07:08:05.944 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:05.944 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:0 cluster instance 2020-04-10T07:08:05.945 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:0 2020-04-10T07:08:05.945 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:08:05.945 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:05.945 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:08:05.945 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:08:05.945 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster2:0 ] 2020-04-10T07:08:05.945 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster2 2020-04-10T07:08:05.946 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:08:05.946 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:08:05.946 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:08:05.946 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 2020-04-10T07:08:05.946 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd-mirror --cluster cluster2 --id mirror.0 --rbd-mirror-delete-retry-interval=5 --rbd-mirror-image-state-check-interval=5 --rbd-mirror-journal-poll-age=1 --rbd-mirror-pool-replayers-refresh-interval=5 --debug-rbd=30 --debug-journaler=30 --debug-rbd_mirror=30 --daemonize=true 2020-04-10T07:08:05.967 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ start_mirror cluster2:1 2020-04-10T07:08:05.967 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2:1 2020-04-10T07:08:05.968 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:05.968 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:1 cluster instance 2020-04-10T07:08:05.968 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:1 2020-04-10T07:08:05.968 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:08:05.968 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:05.968 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:08:05.968 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:08:05.969 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster2:1 ] 2020-04-10T07:08:05.969 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster2 2020-04-10T07:08:05.969 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:08:05.969 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:08:05.969 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:08:05.969 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 2020-04-10T07:08:05.969 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd-mirror --cluster cluster2 --id mirror.1 --rbd-mirror-delete-retry-interval=5 --rbd-mirror-image-state-check-interval=5 --rbd-mirror-journal-poll-age=1 --rbd-mirror-pool-replayers-refresh-interval=5 --debug-rbd=30 --debug-journaler=30 --debug-rbd_mirror=30 --daemonize=true 2020-04-10T07:08:05.990 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ start_mirror cluster2:2 2020-04-10T07:08:05.991 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2:2 2020-04-10T07:08:05.991 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:05.991 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:2 cluster instance 2020-04-10T07:08:05.991 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:2 2020-04-10T07:08:05.991 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:08:05.991 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:05.991 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:08:05.992 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:08:05.992 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster2:2 ] 2020-04-10T07:08:05.992 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster2 2020-04-10T07:08:05.992 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:08:05.992 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:08:05.992 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:08:05.992 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 2020-04-10T07:08:05.993 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd-mirror --cluster cluster2 --id mirror.2 --rbd-mirror-delete-retry-interval=5 --rbd-mirror-image-state-check-interval=5 --rbd-mirror-journal-poll-age=1 --rbd-mirror-pool-replayers-refresh-interval=5 --debug-rbd=30 --debug-journaler=30 --debug-rbd_mirror=30 --daemonize=true 2020-04-10T07:08:06.023 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ start_mirror cluster2:3 2020-04-10T07:08:06.023 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2:3 2020-04-10T07:08:06.023 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:06.023 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:3 cluster instance 2020-04-10T07:08:06.023 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:3 2020-04-10T07:08:06.024 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:08:06.024 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:06.024 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:08:06.024 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:08:06.024 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 3 = cluster2:3 ] 2020-04-10T07:08:06.024 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster2 2020-04-10T07:08:06.024 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:08:06.025 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=3 2020-04-10T07:08:06.025 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:08:06.025 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 2020-04-10T07:08:06.025 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd-mirror --cluster cluster2 --id mirror.3 --rbd-mirror-delete-retry-interval=5 --rbd-mirror-image-state-check-interval=5 --rbd-mirror-journal-poll-age=1 --rbd-mirror-pool-replayers-refresh-interval=5 --debug-rbd=30 --debug-journaler=30 --debug-rbd_mirror=30 --daemonize=true 2020-04-10T07:08:06.060 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ demote_image cluster2 mirror test 2020-04-10T07:08:06.060 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:08:06.060 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:06.061 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:06.061 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster2 mirror image demote mirror/test 2020-04-10T07:08:07.149 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Image demoted to non-primary 2020-04-10T07:08:07.156 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_stopped cluster1 mirror test 2020-04-10T07:08:07.157 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:07.157 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:07.157 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:07.157 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_state cluster1 mirror test stopped 2020-04-10T07:08:07.157 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:07.157 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:07.157 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:07.158 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state=stopped 2020-04-10T07:08:07.158 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local s 2020-04-10T07:08:07.158 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:08:08.157 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_image_replay_state cluster1 mirror test stopped 2020-04-10T07:08:08.158 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:08.158 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:08.158 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:08.158 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local test_state=stopped 2020-04-10T07:08:08.158 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_result 2020-04-10T07:08:08.158 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_state=stopped 2020-04-10T07:08:08.160 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep -i state 2020-04-10T07:08:08.160 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2020-04-10T07:08:08.160 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_instance=cluster1 2020-04-10T07:08:08.160 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:08:08.160 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:08.160 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance=cluster1 2020-04-10T07:08:08.161 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local loop_instance 2020-04-10T07:08:08.161 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0 2020-04-10T07:08:08.161 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ cluster1 != cluster1 ] 2020-04-10T07:08:08.161 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:08:08.162 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2020-04-10T07:08:08.163 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:08:08.163 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:08:08.163 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:08.163 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:08:08.163 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:08:08.163 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:08:08.164 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:08.164 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:08.164 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:08:08.166 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:08:08.167 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:08:08.167 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:08.167 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:08:08.167 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:08:08.167 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster1 2020-04-10T07:08:08.167 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:08:08.168 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:08.168 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:08.168 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:08:08.168 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:08:08.168 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:08:08.168 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:08.168 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:08.169 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:08:08.169 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:08:08.169 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:08:08.169 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:08:08.169 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:08:08.169 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:08:08.169 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:08:08.179 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:08:08.179 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:08:08.179 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:08:08.179 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2020-04-10T07:08:08.269 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:no valid command found; 1 closest matches: 2020-04-10T07:08:08.270 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status mirror/test1 2020-04-10T07:08:08.270 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:08:08.272 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2020-04-10T07:08:08.272 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:1 2020-04-10T07:08:08.272 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:08:08.273 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:08.273 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 cluster instance 2020-04-10T07:08:08.273 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:08:08.273 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:08:08.273 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:08.273 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:08.273 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:08:08.273 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:08:08.274 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:08:08.274 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:08.274 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:08:08.274 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:08:08.274 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:1 cluster1 2020-04-10T07:08:08.274 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:1 2020-04-10T07:08:08.274 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:08.275 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:08.275 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2020-04-10T07:08:08.275 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:08:08.275 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:08:08.275 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:08.275 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:08.275 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:08:08.276 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:08:08.276 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:08:08.276 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:08:08.276 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:08:08.276 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:08:08.276 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2020-04-10T07:08:08.293 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:08:08.293 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:08:08.294 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:08:08.294 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2020-04-10T07:08:08.373 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:Invalid command: unused arguments: ['mirror/test'] 2020-04-10T07:08:08.373 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status : get status for rbd mirror 2020-04-10T07:08:08.373 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:08:08.376 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:2 rbd mirror status mirror/test 2020-04-10T07:08:08.376 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:2 2020-04-10T07:08:08.376 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:08:08.376 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:08.376 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 cluster instance 2020-04-10T07:08:08.377 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:08:08.377 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:08:08.377 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:08.377 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:08.377 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:08:08.377 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:08:08.377 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:08:08.378 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:08.378 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:08:08.378 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:08:08.378 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:2 cluster1 2020-04-10T07:08:08.378 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:2 2020-04-10T07:08:08.378 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:08.378 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:08.378 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2020-04-10T07:08:08.379 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:08:08.379 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:08:08.380 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:08.380 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:08.380 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:08:08.380 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:08:08.380 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:08:08.380 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:08:08.380 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:08:08.381 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:08:08.381 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2020-04-10T07:08:08.398 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:08:08.398 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:08:08.398 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:08:08.398 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2020-04-10T07:08:08.493 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:08:08.494 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ status_result= "state": "Stopped" 2020-04-10T07:08:08.495 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo "state": "Stopped" 2020-04-10T07:08:08.495 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep -i Replaying 2020-04-10T07:08:08.497 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test stopped = stopped 2020-04-10T07:08:08.497 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:08:08.497 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+unknown 2020-04-10T07:08:08.498 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:08.498 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:08.498 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:08.498 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+unknown 2020-04-10T07:08:08.498 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:08:08.498 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:08:08.498 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:08:09.498 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_status_in_pool_dir cluster1 mirror test up+unknown 2020-04-10T07:08:09.499 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:09.499 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:09.499 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:09.499 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+unknown 2020-04-10T07:08:09.499 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:08:09.499 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:08:09.500 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror-test.mirror_status 2020-04-10T07:08:09.500 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror-test.mirror_status 2020-04-10T07:08:09.500 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:08:09.501 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:08:09.501 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test 2020-04-10T07:08:09.501 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ tee /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:08:09.564 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:test: 2020-04-10T07:08:09.564 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: global_id: a9af7d9e-1404-4730-9e18-c0418a915346 2020-04-10T07:08:09.564 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+unknown 2020-04-10T07:08:09.564 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: remote image demoted 2020-04-10T07:08:09.564 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: service: mirror.2 on smithi204 2020-04-10T07:08:09.565 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:08:09 2020-04-10T07:08:09.565 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: peer_sites: 2020-04-10T07:08:09.565 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: name: cluster2-DNE 2020-04-10T07:08:09.565 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+unknown 2020-04-10T07:08:09.565 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: remote image demoted 2020-04-10T07:08:09.565 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:08:08 2020-04-10T07:08:09.572 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ state: .*up+unknown /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:08:09.574 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+unknown 2020-04-10T07:08:09.574 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ description: .* /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:08:09.575 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: remote image demoted 2020-04-10T07:08:09.575 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:08:09.576 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+unknown 2020-04-10T07:08:09.576 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^up+ 2020-04-10T07:08:09.577 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+unknown 2020-04-10T07:08:09.577 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep service: *mirror..* on /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:08:09.578 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: service: mirror.2 on smithi204 2020-04-10T07:08:09.579 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:08:09.582 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update=2020-04-10 07:08:09 2020-04-10T07:08:09.582 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+unknown 2020-04-10 07:08:09 2020-04-10T07:08:09.582 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:09.582 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:09.582 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:09.582 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+unknown 2020-04-10T07:08:09.583 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local prev_last_update=2020-04-10 07:08:09 2020-04-10T07:08:09.583 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror.mirror_status 2020-04-10T07:08:09.583 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror.mirror_status 2020-04-10T07:08:09.583 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:08:09.584 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror.mirror_status 2020-04-10T07:08:09.584 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2020-04-10T07:08:09.629 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: failed to query services: (13) Permission denied 2020-04-10T07:08:09.677 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update state 2020-04-10T07:08:09.677 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2020-04-10T07:08:09.683 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ last_update=2020-04-10 07:08:09 2020-04-10T07:08:09.683 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2020-04-10T07:08:09.689 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ state=up+unknown 2020-04-10T07:08:09.689 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+unknown 2020-04-10T07:08:09.689 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep up+unknown 2020-04-10T07:08:09.690 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+unknown 2020-04-10T07:08:09.691 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:08:09.691 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:08:09.691 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_status_in_pool_dir cluster2 mirror test up+unknown 2020-04-10T07:08:09.691 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:08:09.691 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:09.692 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:09.692 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+unknown 2020-04-10T07:08:09.692 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:08:09.692 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:08:09.692 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:08:10.693 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_status_in_pool_dir cluster2 mirror test up+unknown 2020-04-10T07:08:10.693 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:08:10.693 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:10.693 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:10.694 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+unknown 2020-04-10T07:08:10.694 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:08:10.694 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:08:10.694 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster2-mirror-test.mirror_status 2020-04-10T07:08:10.694 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2-mirror-test.mirror_status 2020-04-10T07:08:10.694 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:08:10.695 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:08:10.695 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster2 mirror image status mirror/test 2020-04-10T07:08:10.696 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ tee /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:08:10.766 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:test: 2020-04-10T07:08:10.766 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: global_id: a9af7d9e-1404-4730-9e18-c0418a915346 2020-04-10T07:08:10.766 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+unknown 2020-04-10T07:08:10.766 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: remote image demoted 2020-04-10T07:08:10.767 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:08:08 2020-04-10T07:08:10.767 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: peer_sites: 2020-04-10T07:08:10.767 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: name: cluster1-DNE 2020-04-10T07:08:10.767 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+unknown 2020-04-10T07:08:10.767 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: remote image demoted 2020-04-10T07:08:10.767 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:08:09 2020-04-10T07:08:10.772 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ state: .*up+unknown /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:08:10.773 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+unknown 2020-04-10T07:08:10.774 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ description: .* /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:08:10.775 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: remote image demoted 2020-04-10T07:08:10.775 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:08:10.775 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+unknown 2020-04-10T07:08:10.775 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^up+ 2020-04-10T07:08:10.776 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+unknown 2020-04-10T07:08:10.776 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep service: *mirror..* on /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:08:10.777 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 1 2020-04-10T07:08:10.777 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 2 2020-04-10T07:08:12.779 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_status_in_pool_dir cluster2 mirror test up+unknown 2020-04-10T07:08:12.779 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:08:12.779 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:12.779 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:12.779 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+unknown 2020-04-10T07:08:12.779 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:08:12.780 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:08:12.780 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster2-mirror-test.mirror_status 2020-04-10T07:08:12.780 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2-mirror-test.mirror_status 2020-04-10T07:08:12.781 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:08:12.781 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:08:12.782 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster2 mirror image status mirror/test 2020-04-10T07:08:12.782 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ tee /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:08:12.850 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:test: 2020-04-10T07:08:12.850 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: global_id: a9af7d9e-1404-4730-9e18-c0418a915346 2020-04-10T07:08:12.850 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+unknown 2020-04-10T07:08:12.850 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: remote image demoted 2020-04-10T07:08:12.850 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: service: mirror.1 on smithi204 2020-04-10T07:08:12.851 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:08:11 2020-04-10T07:08:12.851 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: peer_sites: 2020-04-10T07:08:12.851 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: name: cluster1-DNE 2020-04-10T07:08:12.851 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+unknown 2020-04-10T07:08:12.851 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: remote image demoted 2020-04-10T07:08:12.851 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:08:09 2020-04-10T07:08:12.858 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ state: .*up+unknown /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:08:12.860 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+unknown 2020-04-10T07:08:12.860 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ description: .* /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:08:12.861 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: remote image demoted 2020-04-10T07:08:12.861 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:08:12.862 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ + echo up+unknown 2020-04-10T07:08:12.862 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:grep ^up+ 2020-04-10T07:08:12.863 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+unknown 2020-04-10T07:08:12.863 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep service: *mirror..* on /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:08:12.864 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: service: mirror.1 on smithi204 2020-04-10T07:08:12.865 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:08:12.867 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update=2020-04-10 07:08:11 2020-04-10T07:08:12.867 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_mirror_pool_status_verbose cluster2 mirror test up+unknown 2020-04-10 07:08:11 2020-04-10T07:08:12.867 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:08:12.867 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:12.867 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:12.868 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+unknown 2020-04-10T07:08:12.868 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local prev_last_update=2020-04-10 07:08:11 2020-04-10T07:08:12.868 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster2-mirror.mirror_status 2020-04-10T07:08:12.868 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2-mirror.mirror_status 2020-04-10T07:08:12.868 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:08:12.869 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster2-mirror.mirror_status 2020-04-10T07:08:12.869 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2020-04-10T07:08:12.906 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: failed to query services: (13) Permission denied 2020-04-10T07:08:12.946 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update state 2020-04-10T07:08:12.946 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2020-04-10T07:08:12.949 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ last_update=2020-04-10 07:08:11 2020-04-10T07:08:12.949 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2020-04-10T07:08:12.952 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ state=up+unknown 2020-04-10T07:08:12.952 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ + grep up+unknown 2020-04-10T07:08:12.952 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:echo up+unknown 2020-04-10T07:08:12.953 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+unknown 2020-04-10T07:08:12.953 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:08:12.954 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:08:12.954 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ promote_image cluster2 mirror test 2020-04-10T07:08:12.954 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:08:12.954 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:12.954 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:12.954 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local force= 2020-04-10T07:08:12.954 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/test 2020-04-10T07:08:13.094 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Image promoted to primary 2020-04-10T07:08:13.103 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_started cluster1 mirror test 2020-04-10T07:08:13.103 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:13.103 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:13.103 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:13.103 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2020-04-10T07:08:13.104 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:13.104 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:13.104 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:13.104 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state=started 2020-04-10T07:08:13.104 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local s 2020-04-10T07:08:13.104 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:08:14.105 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_image_replay_state cluster1 mirror test started 2020-04-10T07:08:14.105 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:14.105 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:14.105 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:14.105 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local test_state=started 2020-04-10T07:08:14.106 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_result 2020-04-10T07:08:14.106 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_state=stopped 2020-04-10T07:08:14.106 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2020-04-10T07:08:14.106 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_instance=cluster1 2020-04-10T07:08:14.106 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:08:14.106 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ + local cluster=cluster1 2020-04-10T07:08:14.106 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance=cluster1 2020-04-10T07:08:14.107 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ greplocal loop_instance -i 2020-04-10T07:08:14.107 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state 2020-04-10T07:08:14.107 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0 2020-04-10T07:08:14.109 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ cluster1 != cluster1 ] 2020-04-10T07:08:14.109 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:08:14.109 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2020-04-10T07:08:14.109 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:08:14.109 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:08:14.109 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:14.109 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:08:14.110 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:08:14.110 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:08:14.110 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:14.110 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:14.110 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:08:14.110 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:08:14.110 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:08:14.111 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:14.111 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:08:14.111 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:08:14.113 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster1 2020-04-10T07:08:14.113 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:08:14.113 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:14.113 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:14.113 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:08:14.114 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:08:14.114 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:08:14.114 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:14.114 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:14.114 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:08:14.114 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:08:14.114 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:08:14.115 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:08:14.115 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:08:14.115 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:08:14.115 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:08:14.131 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:08:14.131 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:08:14.131 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:08:14.132 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2020-04-10T07:08:14.228 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:no valid command found; 1 closest matches: 2020-04-10T07:08:14.228 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status mirror/test1 2020-04-10T07:08:14.228 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:08:14.231 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2020-04-10T07:08:14.231 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:1 2020-04-10T07:08:14.231 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:08:14.231 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:14.231 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 cluster instance 2020-04-10T07:08:14.231 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:08:14.231 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:08:14.232 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:14.232 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:14.232 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:08:14.232 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:08:14.232 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:08:14.232 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:14.232 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:08:14.233 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:08:14.233 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:1 cluster1 2020-04-10T07:08:14.233 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:1 2020-04-10T07:08:14.233 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:14.233 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:14.233 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2020-04-10T07:08:14.233 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:08:14.233 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:08:14.234 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:14.234 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:14.234 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:08:14.234 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:08:14.234 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:08:14.234 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:08:14.234 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:08:14.235 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:08:14.235 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2020-04-10T07:08:14.250 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:08:14.250 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:08:14.250 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:08:14.250 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2020-04-10T07:08:14.337 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:Invalid command: unused arguments: ['mirror/test'] 2020-04-10T07:08:14.338 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status : get status for rbd mirror 2020-04-10T07:08:14.338 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:08:14.340 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:2 rbd mirror status mirror/test 2020-04-10T07:08:14.340 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:2 2020-04-10T07:08:14.340 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:08:14.340 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:14.341 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 cluster instance 2020-04-10T07:08:14.341 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:08:14.341 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:08:14.341 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:14.341 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:14.341 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:08:14.341 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:08:14.342 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:08:14.342 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:14.342 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:08:14.342 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:08:14.342 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:2 cluster1 2020-04-10T07:08:14.342 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:2 2020-04-10T07:08:14.342 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:14.342 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:14.343 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2020-04-10T07:08:14.343 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:08:14.343 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:08:14.343 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:14.343 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:14.343 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:08:14.343 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:08:14.343 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:08:14.344 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:08:14.344 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:08:14.344 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:08:14.344 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2020-04-10T07:08:14.362 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:08:14.362 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:08:14.362 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:08:14.362 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2020-04-10T07:08:14.461 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:08:14.462 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ status_result= "state": "Starting" 2020-04-10T07:08:14.462 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo "state": "Starting" 2020-04-10T07:08:14.463 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep -i Replaying 2020-04-10T07:08:14.463 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test started = stopped 2020-04-10T07:08:14.463 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 2 2020-04-10T07:08:16.464 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_image_replay_state cluster1 mirror test started 2020-04-10T07:08:16.465 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:16.465 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:16.465 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:16.465 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local test_state=started 2020-04-10T07:08:16.465 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_result 2020-04-10T07:08:16.465 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_state=stopped 2020-04-10T07:08:16.467 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2020-04-10T07:08:16.467 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_instance=cluster1 2020-04-10T07:08:16.467 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:08:16.467 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:16.467 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ + local instance=cluster1 2020-04-10T07:08:16.467 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local loop_instance 2020-04-10T07:08:16.467 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:grep -i state 2020-04-10T07:08:16.468 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0 2020-04-10T07:08:16.468 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ cluster1 != cluster1 ] 2020-04-10T07:08:16.468 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:08:16.470 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2020-04-10T07:08:16.470 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:08:16.470 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:08:16.471 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:16.471 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:08:16.471 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:08:16.471 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:08:16.471 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:16.471 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:16.471 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:08:16.472 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:08:16.472 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:08:16.472 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:16.472 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:08:16.472 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:08:16.472 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster1 2020-04-10T07:08:16.472 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:08:16.473 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:16.473 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:16.473 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:08:16.473 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:08:16.473 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:08:16.473 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:16.473 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:16.474 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:08:16.474 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:08:16.474 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:08:16.474 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:08:16.474 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:08:16.474 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:08:16.474 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:08:16.489 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:08:16.489 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:08:16.489 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:08:16.490 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2020-04-10T07:08:16.582 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:no valid command found; 1 closest matches: 2020-04-10T07:08:16.583 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status mirror/test1 2020-04-10T07:08:16.583 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:08:16.587 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2020-04-10T07:08:16.587 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:1 2020-04-10T07:08:16.588 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:08:16.588 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:16.588 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 cluster instance 2020-04-10T07:08:16.588 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:08:16.588 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:08:16.588 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:16.588 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:16.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:08:16.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:08:16.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:08:16.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:16.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:08:16.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:08:16.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:1 cluster1 2020-04-10T07:08:16.590 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:1 2020-04-10T07:08:16.590 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:16.590 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:16.590 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2020-04-10T07:08:16.590 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:08:16.590 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:08:16.590 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:16.591 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:16.591 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:08:16.591 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:08:16.591 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:08:16.591 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:08:16.591 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:08:16.591 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:08:16.591 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2020-04-10T07:08:16.607 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:08:16.607 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:08:16.607 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:08:16.607 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2020-04-10T07:08:16.692 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:Invalid command: unused arguments: ['mirror/test'] 2020-04-10T07:08:16.692 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status : get status for rbd mirror 2020-04-10T07:08:16.692 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:08:16.697 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:2 rbd mirror status mirror/test 2020-04-10T07:08:16.697 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:2 2020-04-10T07:08:16.697 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:08:16.698 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:16.698 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 cluster instance 2020-04-10T07:08:16.698 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:08:16.698 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:08:16.698 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:16.698 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:16.698 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:08:16.699 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:08:16.699 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:08:16.699 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:16.699 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:08:16.699 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:08:16.699 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:2 cluster1 2020-04-10T07:08:16.699 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:2 2020-04-10T07:08:16.700 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:16.700 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:16.700 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2020-04-10T07:08:16.700 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:08:16.700 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:08:16.700 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:16.700 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:16.700 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:08:16.701 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:08:16.701 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:08:16.701 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:08:16.701 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:08:16.701 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:08:16.701 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2020-04-10T07:08:16.725 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:08:16.726 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:08:16.726 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:08:16.726 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2020-04-10T07:08:16.815 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:08:16.815 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ status_result= "state": "Replaying" 2020-04-10T07:08:16.815 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ + echogrep "state": "Replaying" -i 2020-04-10T07:08:16.816 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: Replaying 2020-04-10T07:08:16.817 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "state": "Replaying" 2020-04-10T07:08:16.817 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ current_state=started 2020-04-10T07:08:16.817 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test started = started 2020-04-10T07:08:16.817 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:08:16.818 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ write_image cluster2 mirror test 100 2020-04-10T07:08:16.818 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:08:16.818 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:16.818 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:16.818 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local count=100 2020-04-10T07:08:16.818 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local size= 2020-04-10T07:08:16.818 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 2020-04-10T07:08:16.819 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ size=4096 2020-04-10T07:08:16.819 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 bench mirror/test --io-type write --io-size 4096 --io-threads 1 --io-total 409600 --io-pattern rand 2020-04-10T07:08:16.904 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:bench type write io_size 4096 io_threads 1 bytes 409600 pattern random 2020-04-10T07:08:16.969 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: SEC OPS OPS/SEC BYTES/SEC 2020-04-10T07:08:16.970 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:elapsed: 0 ops: 100 ops/sec: 1562.5 bytes/sec: 6.1 MiB/s 2020-04-10T07:08:16.982 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror test 2020-04-10T07:08:16.982 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1 2020-04-10T07:08:16.982 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:08:16.982 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:16.983 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:16.983 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ snapshot = journal ] 2020-04-10T07:08:16.983 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ snapshot = snapshot ] 2020-04-10T07:08:16.983 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror test 2020-04-10T07:08:16.983 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1 2020-04-10T07:08:16.983 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:08:16.983 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:16.984 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:16.984 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster2-mirror-test.status 2020-04-10T07:08:16.984 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2-mirror-test.status 2020-04-10T07:08:16.984 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:08:16.985 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.status 2020-04-10T07:08:16.985 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror-test.status 2020-04-10T07:08:16.986 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror-test.status 2020-04-10T07:08:16.986 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:08:16.986 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.status 2020-04-10T07:08:16.986 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mirror_image_snapshot cluster2 mirror test 2020-04-10T07:08:16.986 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:08:16.986 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:16.986 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:16.986 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test 2020-04-10T07:08:17.118 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Snapshot ID: 21 2020-04-10T07:08:17.128 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_newest_mirror_snapshot cluster2 mirror test /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.status 2020-04-10T07:08:17.128 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:08:17.128 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:17.128 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:17.128 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local log=/tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.status 2020-04-10T07:08:17.129 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 snap list --all mirror/test --format xml 2020-04-10T07:08:17.129 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2020-04-10T07:08:17.180 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //snapshot/id 2020-04-10T07:08:17.185 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snapshot_id=21 2020-04-10T07:08:17.185 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ true 2020-04-10T07:08:17.185 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0.2 2020-04-10T07:08:17.386 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.status 2020-04-10T07:08:17.387 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:17.387 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:17.387 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:17.387 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.status 2020-04-10T07:08:17.387 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2020-04-10T07:08:17.387 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2020-04-10T07:08:17.456 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2020-04-10T07:08:17.461 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local primary_snapshot_id=19 2020-04-10T07:08:17.462 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test 21 = 19 2020-04-10T07:08:17.462 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0.4 2020-04-10T07:08:17.863 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.status 2020-04-10T07:08:17.863 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:17.863 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:17.863 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:17.864 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.status 2020-04-10T07:08:17.864 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2020-04-10T07:08:17.864 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2020-04-10T07:08:17.931 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2020-04-10T07:08:17.936 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local primary_snapshot_id=19 2020-04-10T07:08:17.936 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test 21 = 19 2020-04-10T07:08:17.936 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0.8 2020-04-10T07:08:18.739 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.status 2020-04-10T07:08:18.740 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:18.740 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:18.740 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:18.740 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.status 2020-04-10T07:08:18.740 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2020-04-10T07:08:18.740 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2020-04-10T07:08:18.810 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2020-04-10T07:08:18.816 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local primary_snapshot_id=21 2020-04-10T07:08:18.816 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test 21 = 21 2020-04-10T07:08:18.817 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:08:18.817 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_status_in_pool_dir cluster2 mirror test up+stopped 2020-04-10T07:08:18.817 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:08:18.817 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:18.817 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:18.817 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+stopped 2020-04-10T07:08:18.817 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:08:18.818 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:08:18.818 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:08:19.818 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_status_in_pool_dir cluster2 mirror test up+stopped 2020-04-10T07:08:19.818 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:08:19.818 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:19.818 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:19.819 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+stopped 2020-04-10T07:08:19.819 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:08:19.819 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:08:19.819 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster2-mirror-test.mirror_status 2020-04-10T07:08:19.820 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2-mirror-test.mirror_status 2020-04-10T07:08:19.820 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:08:19.820 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:08:19.821 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster2 mirror image+ status mirror/test 2020-04-10T07:08:19.821 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:tee /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:08:19.892 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:test: 2020-04-10T07:08:19.893 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: global_id: a9af7d9e-1404-4730-9e18-c0418a915346 2020-04-10T07:08:19.893 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+stopped 2020-04-10T07:08:19.893 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: local image is primary 2020-04-10T07:08:19.893 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: service: mirror.1 on smithi204 2020-04-10T07:08:19.893 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:08:16 2020-04-10T07:08:19.893 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: peer_sites: 2020-04-10T07:08:19.893 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: name: cluster1-DNE 2020-04-10T07:08:19.894 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+replaying 2020-04-10T07:08:19.894 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: replaying, {"replay_state": "idle", "remote_snapshot_timestamp": 1586502493, "local_snapshot_timestamp": 1586502487} 2020-04-10T07:08:19.894 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:08:14 2020-04-10T07:08:19.894 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: snapshots: 2020-04-10T07:08:19.894 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: 17 .mirror.primary.a9af7d9e-1404-4730-9e18-c0418a915346.ae5a942a-32bb-40d8-b4b6-4b6da2b8780c (peer_uuids:[90b6cfa2-5481-4edf-b462-5494aab62f6f]) 2020-04-10T07:08:19.894 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: 21 .mirror.primary.a9af7d9e-1404-4730-9e18-c0418a915346.720d033f-30db-4fdf-9736-fd21d1dfaf57 (peer_uuids:[90b6cfa2-5481-4edf-b462-5494aab62f6f]) 2020-04-10T07:08:19.898 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ state: .*up+stopped /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:08:19.899 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+stopped 2020-04-10T07:08:19.899 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ description: .* /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:08:19.900 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: local image is primary 2020-04-10T07:08:19.900 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:08:19.901 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ + grep ^up+ 2020-04-10T07:08:19.901 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:echo up+stopped 2020-04-10T07:08:19.902 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep service: *mirror..* on /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:08:19.902 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+stopped 2020-04-10T07:08:19.902 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: service: mirror.1 on smithi204 2020-04-10T07:08:19.902 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:08:19.904 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update=2020-04-10 07:08:16 2020-04-10T07:08:19.904 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_mirror_pool_status_verbose cluster2 mirror test up+stopped 2020-04-10 07:08:16 2020-04-10T07:08:19.904 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:08:19.905 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:19.905 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:19.905 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+stopped 2020-04-10T07:08:19.905 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local prev_last_update=2020-04-10 07:08:16 2020-04-10T07:08:19.905 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster2-mirror.mirror_status 2020-04-10T07:08:19.905 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2-mirror.mirror_status 2020-04-10T07:08:19.905 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:08:19.906 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster2-mirror.mirror_status 2020-04-10T07:08:19.906 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2020-04-10T07:08:19.945 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: failed to query services: (13) Permission denied 2020-04-10T07:08:19.987 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update state 2020-04-10T07:08:19.988 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2020-04-10T07:08:19.990 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ last_update=2020-04-10 07:08:16 2020-04-10T07:08:19.990 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2020-04-10T07:08:19.993 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ state=up+stopped 2020-04-10T07:08:19.993 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+stopped 2020-04-10T07:08:19.993 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep up+stopped 2020-04-10T07:08:19.994 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+stopped 2020-04-10T07:08:19.995 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:08:19.995 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:08:19.995 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2020-04-10T07:08:19.995 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:19.995 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:19.996 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:19.996 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:08:19.996 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:08:19.996 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:08:19.996 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:08:20.995 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying 2020-04-10T07:08:20.996 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:20.996 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:20.996 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:20.996 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:08:20.996 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:08:20.996 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:08:20.996 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror-test.mirror_status 2020-04-10T07:08:20.997 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror-test.mirror_status 2020-04-10T07:08:20.997 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:08:20.998 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:08:20.998 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test 2020-04-10T07:08:20.998 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ tee /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:08:21.075 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:test: 2020-04-10T07:08:21.076 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: global_id: a9af7d9e-1404-4730-9e18-c0418a915346 2020-04-10T07:08:21.076 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+replaying 2020-04-10T07:08:21.076 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: replaying, {"replay_state": "idle", "remote_snapshot_timestamp": 1586502493, "local_snapshot_timestamp": 1586502487} 2020-04-10T07:08:21.076 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: service: mirror.2 on smithi204 2020-04-10T07:08:21.076 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:08:14 2020-04-10T07:08:21.076 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: peer_sites: 2020-04-10T07:08:21.076 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: name: cluster2-DNE 2020-04-10T07:08:21.077 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+stopped 2020-04-10T07:08:21.077 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: local image is primary 2020-04-10T07:08:21.077 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:08:16 2020-04-10T07:08:21.086 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ state: .*up+replaying /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:08:21.086 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ description: .* /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:08:21.086 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:08:21.086 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ + echo up+replaying 2020-04-10T07:08:21.087 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:grep ^up+ 2020-04-10T07:08:21.087 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep service: *mirror..* on /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:08:21.087 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+replaying 2020-04-10T07:08:21.087 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: replaying, {"replay_state": "idle", "remote_snapshot_timestamp": 1586502493, "local_snapshot_timestamp": 1586502487} 2020-04-10T07:08:21.087 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+replaying 2020-04-10T07:08:21.089 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: service: mirror.2 on smithi204 2020-04-10T07:08:21.089 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:08:21.089 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update=2020-04-10 07:08:14 2020-04-10T07:08:21.089 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying 2020-04-10 07:08:14 2020-04-10T07:08:21.089 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:21.090 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:21.090 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:21.090 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:08:21.090 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local prev_last_update=2020-04-10 07:08:14 2020-04-10T07:08:21.090 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror.mirror_status 2020-04-10T07:08:21.090 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror.mirror_status 2020-04-10T07:08:21.090 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:08:21.091 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror.mirror_status 2020-04-10T07:08:21.091 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2020-04-10T07:08:21.133 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: failed to query services: (13) Permission denied 2020-04-10T07:08:21.179 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update state 2020-04-10T07:08:21.179 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2020-04-10T07:08:21.191 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ last_update=2020-04-10 07:08:14 2020-04-10T07:08:21.191 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2020-04-10T07:08:21.191 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ state=up+replaying 2020-04-10T07:08:21.191 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+replaying 2020-04-10T07:08:21.192 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep up+replaying 2020-04-10T07:08:21.192 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:08:21.192 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:08:21.192 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ compare_images mirror test 2020-04-10T07:08:21.192 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:21.192 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:21.193 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local ret=0 2020-04-10T07:08:21.193 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster2-mirror-test.export 2020-04-10T07:08:21.193 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2-mirror-test.export 2020-04-10T07:08:21.193 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:08:21.193 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+replaying 2020-04-10T07:08:21.194 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local rmt_export=/tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.export 2020-04-10T07:08:21.194 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror-test.export 2020-04-10T07:08:21.195 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror-test.export 2020-04-10T07:08:21.195 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:08:21.195 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local loc_export=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.export 2020-04-10T07:08:21.195 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rm -f /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.export /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.export 2020-04-10T07:08:21.196 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 export mirror/test /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.export 2020-04-10T07:08:21.365 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2020-04-10T07:08:21.368 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 export mirror/test /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.export 2020-04-10T07:08:21.539 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2020-04-10T07:08:21.542 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cmp /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.export /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.export 2020-04-10T07:08:21.621 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rm -f /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.export /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.export 2020-04-10T07:08:21.649 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:08:21.650 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ demote_image cluster2 mirror test 2020-04-10T07:08:21.650 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:08:21.650 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:21.650 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:21.650 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster2 mirror image demote mirror/test 2020-04-10T07:08:22.155 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Image demoted to non-primary 2020-04-10T07:08:22.162 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_stopped cluster1 mirror test 2020-04-10T07:08:22.162 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:22.162 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:22.163 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:22.163 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_state cluster1 mirror test stopped 2020-04-10T07:08:22.163 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:22.163 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:22.163 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:22.163 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state=stopped 2020-04-10T07:08:22.163 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local s 2020-04-10T07:08:22.164 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:08:23.164 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_image_replay_state cluster1 mirror test stopped 2020-04-10T07:08:23.164 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:23.164 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:23.164 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:23.165 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local test_state=stopped 2020-04-10T07:08:23.165 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_result 2020-04-10T07:08:23.165 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_state=stopped 2020-04-10T07:08:23.165 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2020-04-10T07:08:23.165 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_instance=cluster1 2020-04-10T07:08:23.165 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:08:23.166 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:23.166 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ + local instance=cluster1 2020-04-10T07:08:23.166 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local loop_instance 2020-04-10T07:08:23.166 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:grep -i state 2020-04-10T07:08:23.166 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0 2020-04-10T07:08:23.168 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ cluster1 != cluster1 ] 2020-04-10T07:08:23.168 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:08:23.168 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2020-04-10T07:08:23.168 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:08:23.168 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:08:23.169 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:23.169 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:08:23.169 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:08:23.169 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:08:23.169 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:23.169 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:23.169 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:08:23.170 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:08:23.170 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:08:23.170 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:23.170 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:08:23.170 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:08:23.170 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster1 2020-04-10T07:08:23.170 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:08:23.172 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:23.172 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:23.172 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:08:23.172 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:08:23.173 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:08:23.173 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:23.173 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:23.173 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:08:23.173 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:08:23.173 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:08:23.173 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:08:23.174 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:08:23.174 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:08:23.174 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:08:23.188 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:08:23.188 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:08:23.188 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:08:23.188 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2020-04-10T07:08:23.282 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:no valid command found; 1 closest matches: 2020-04-10T07:08:23.282 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status mirror/test1 2020-04-10T07:08:23.282 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:08:23.284 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2020-04-10T07:08:23.284 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:1 2020-04-10T07:08:23.284 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:08:23.285 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:23.285 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 cluster instance 2020-04-10T07:08:23.285 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:08:23.285 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:08:23.285 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:23.285 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:23.285 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:08:23.285 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:08:23.286 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:08:23.286 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:23.286 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:08:23.286 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:08:23.286 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:1 cluster1 2020-04-10T07:08:23.286 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:1 2020-04-10T07:08:23.286 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:23.287 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:23.287 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2020-04-10T07:08:23.287 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:08:23.287 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:08:23.287 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:23.287 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:23.287 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:08:23.288 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:08:23.288 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:08:23.288 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:08:23.288 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:08:23.288 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:08:23.288 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2020-04-10T07:08:23.307 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:08:23.307 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:08:23.307 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:08:23.307 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2020-04-10T07:08:23.394 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:Invalid command: unused arguments: ['mirror/test'] 2020-04-10T07:08:23.394 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status : get status for rbd mirror 2020-04-10T07:08:23.394 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:08:23.396 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:2 rbd mirror status mirror/test 2020-04-10T07:08:23.397 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:2 2020-04-10T07:08:23.397 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:08:23.397 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:23.397 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 cluster instance 2020-04-10T07:08:23.397 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:08:23.398 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:08:23.398 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:23.398 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:23.399 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:08:23.399 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:08:23.399 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:08:23.399 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:23.399 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:08:23.399 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:08:23.399 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:2 cluster1 2020-04-10T07:08:23.400 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:2 2020-04-10T07:08:23.400 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:23.400 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:23.400 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2020-04-10T07:08:23.400 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:08:23.400 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:08:23.400 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:23.400 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:23.401 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:08:23.401 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:08:23.401 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:08:23.401 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:08:23.401 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:08:23.401 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:08:23.401 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2020-04-10T07:08:23.417 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:08:23.417 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:08:23.417 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:08:23.417 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2020-04-10T07:08:23.514 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:08:23.515 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ status_result= "state": "Stopped" 2020-04-10T07:08:23.515 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo "state": "Stopped" 2020-04-10T07:08:23.516 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep -i Replaying 2020-04-10T07:08:23.518 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test stopped = stopped 2020-04-10T07:08:23.518 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:08:23.518 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+unknown 2020-04-10T07:08:23.518 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:23.518 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:23.518 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:23.519 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+unknown 2020-04-10T07:08:23.519 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:08:23.519 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:08:23.519 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:08:24.518 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_status_in_pool_dir cluster1 mirror test up+unknown 2020-04-10T07:08:24.519 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:24.519 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:24.519 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:24.519 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+unknown 2020-04-10T07:08:24.519 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:08:24.519 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:08:24.520 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror-test.mirror_status 2020-04-10T07:08:24.520 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror-test.mirror_status 2020-04-10T07:08:24.520 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:08:24.521 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:08:24.521 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test 2020-04-10T07:08:24.521 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ tee /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:08:24.597 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:test: 2020-04-10T07:08:24.598 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: global_id: a9af7d9e-1404-4730-9e18-c0418a915346 2020-04-10T07:08:24.598 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+unknown 2020-04-10T07:08:24.598 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: remote image demoted 2020-04-10T07:08:24.598 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: service: mirror.2 on smithi204 2020-04-10T07:08:24.598 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:08:24 2020-04-10T07:08:24.598 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: peer_sites: 2020-04-10T07:08:24.598 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: name: cluster2-DNE 2020-04-10T07:08:24.599 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+stopped 2020-04-10T07:08:24.599 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: local image is primary 2020-04-10T07:08:24.599 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:08:21 2020-04-10T07:08:24.606 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ state: .*up+unknown /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:08:24.608 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+unknown 2020-04-10T07:08:24.608 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ description: .* /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:08:24.610 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: remote image demoted 2020-04-10T07:08:24.611 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:08:24.611 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo+ up+unknown 2020-04-10T07:08:24.611 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:grep ^up+ 2020-04-10T07:08:24.612 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+unknown 2020-04-10T07:08:24.612 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep service: *mirror..* on /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:08:24.614 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: service: mirror.2 on smithi204 2020-04-10T07:08:24.614 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:08:24.616 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update=2020-04-10 07:08:24 2020-04-10T07:08:24.617 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+unknown 2020-04-10 07:08:24 2020-04-10T07:08:24.617 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:24.617 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:24.617 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:24.617 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+unknown 2020-04-10T07:08:24.617 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local prev_last_update=2020-04-10 07:08:24 2020-04-10T07:08:24.617 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror.mirror_status 2020-04-10T07:08:24.618 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror.mirror_status 2020-04-10T07:08:24.618 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:08:24.619 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror.mirror_status 2020-04-10T07:08:24.619 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2020-04-10T07:08:24.662 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: failed to query services: (13) Permission denied 2020-04-10T07:08:24.716 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update state 2020-04-10T07:08:24.716 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2020-04-10T07:08:24.722 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ last_update=2020-04-10 07:08:24 2020-04-10T07:08:24.722 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2020-04-10T07:08:24.726 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ state=up+unknown 2020-04-10T07:08:24.726 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+unknown 2020-04-10T07:08:24.726 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep up+unknown 2020-04-10T07:08:24.728 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+unknown 2020-04-10T07:08:24.728 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:08:24.729 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:08:24.729 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_status_in_pool_dir cluster2 mirror test up+unknown 2020-04-10T07:08:24.729 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:08:24.729 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:24.729 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:24.729 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+unknown 2020-04-10T07:08:24.729 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:08:24.730 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:08:24.730 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:08:25.730 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_status_in_pool_dir cluster2 mirror test up+unknown 2020-04-10T07:08:25.730 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:08:25.730 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:25.730 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:25.730 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+unknown 2020-04-10T07:08:25.731 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:08:25.731 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:08:25.731 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster2-mirror-test.mirror_status 2020-04-10T07:08:25.731 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2-mirror-test.mirror_status 2020-04-10T07:08:25.731 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:08:25.732 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:08:25.732 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster2 mirror image status mirror/test 2020-04-10T07:08:25.732 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ tee /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:08:25.799 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:test: 2020-04-10T07:08:25.799 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: global_id: a9af7d9e-1404-4730-9e18-c0418a915346 2020-04-10T07:08:25.799 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+stopped 2020-04-10T07:08:25.799 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: local image is primary 2020-04-10T07:08:25.799 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: service: mirror.1 on smithi204 2020-04-10T07:08:25.799 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:08:21 2020-04-10T07:08:25.800 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: peer_sites: 2020-04-10T07:08:25.800 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: name: cluster1-DNE 2020-04-10T07:08:25.800 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+unknown 2020-04-10T07:08:25.800 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: remote image demoted 2020-04-10T07:08:25.800 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:08:24 2020-04-10T07:08:25.805 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ state: .*up+unknown /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:08:25.806 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 1 2020-04-10T07:08:25.806 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 2 2020-04-10T07:08:27.808 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_status_in_pool_dir cluster2 mirror test up+unknown 2020-04-10T07:08:27.808 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:08:27.809 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:27.809 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:27.809 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+unknown 2020-04-10T07:08:27.809 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:08:27.809 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:08:27.809 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster2-mirror-test.mirror_status 2020-04-10T07:08:27.809 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2-mirror-test.mirror_status 2020-04-10T07:08:27.810 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:08:27.810 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:08:27.811 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster2 mirror image status mirror/test 2020-04-10T07:08:27.811 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ tee /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:08:27.873 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:test: 2020-04-10T07:08:27.873 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: global_id: a9af7d9e-1404-4730-9e18-c0418a915346 2020-04-10T07:08:27.873 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+unknown 2020-04-10T07:08:27.873 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: remote image demoted 2020-04-10T07:08:27.874 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: service: mirror.1 on smithi204 2020-04-10T07:08:27.874 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:08:26 2020-04-10T07:08:27.874 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: peer_sites: 2020-04-10T07:08:27.874 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: name: cluster1-DNE 2020-04-10T07:08:27.874 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+unknown 2020-04-10T07:08:27.874 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: remote image demoted 2020-04-10T07:08:27.874 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:08:24 2020-04-10T07:08:27.879 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ state: .*up+unknown /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:08:27.880 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+unknown 2020-04-10T07:08:27.880 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ description: .* /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:08:27.881 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: remote image demoted 2020-04-10T07:08:27.881 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:08:27.882 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ + echo up+unknown 2020-04-10T07:08:27.882 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:grep ^up+ 2020-04-10T07:08:27.882 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep service: *mirror..* on /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:08:27.883 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+unknown 2020-04-10T07:08:27.884 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: service: mirror.1 on smithi204 2020-04-10T07:08:27.884 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:08:27.886 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update=2020-04-10 07:08:26 2020-04-10T07:08:27.886 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_mirror_pool_status_verbose cluster2 mirror test up+unknown 2020-04-10 07:08:26 2020-04-10T07:08:27.886 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:08:27.886 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:27.886 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:27.887 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+unknown 2020-04-10T07:08:27.887 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local prev_last_update=2020-04-10 07:08:26 2020-04-10T07:08:27.887 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster2-mirror.mirror_status 2020-04-10T07:08:27.887 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2-mirror.mirror_status 2020-04-10T07:08:27.888 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:08:27.888 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster2-mirror.mirror_status 2020-04-10T07:08:27.888 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2020-04-10T07:08:27.925 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: failed to query services: (13) Permission denied 2020-04-10T07:08:27.959 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update state 2020-04-10T07:08:27.959 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2020-04-10T07:08:27.962 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ last_update=2020-04-10 07:08:26 2020-04-10T07:08:27.962 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2020-04-10T07:08:27.964 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ state=up+unknown 2020-04-10T07:08:27.964 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+unknown 2020-04-10T07:08:27.964 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep up+unknown 2020-04-10T07:08:27.966 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+unknown 2020-04-10T07:08:27.966 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:08:27.966 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:08:27.966 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ promote_image cluster1 mirror test 2020-04-10T07:08:27.966 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:27.966 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:27.966 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:27.967 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local force= 2020-04-10T07:08:27.967 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster1 mirror image promote mirror/test 2020-04-10T07:08:28.119 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Image promoted to primary 2020-04-10T07:08:28.132 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_started cluster2 mirror test 2020-04-10T07:08:28.132 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:08:28.132 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:28.132 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:28.132 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_state cluster2 mirror test started 2020-04-10T07:08:28.133 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:08:28.133 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:28.133 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:28.133 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state=started 2020-04-10T07:08:28.133 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local s 2020-04-10T07:08:28.133 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:08:29.134 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_image_replay_state cluster2 mirror test started 2020-04-10T07:08:29.134 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:08:29.134 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:29.134 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:29.135 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local test_state=started 2020-04-10T07:08:29.135 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_result 2020-04-10T07:08:29.135 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_state=stopped 2020-04-10T07:08:29.135 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemons cluster2 rbd mirror status mirror/test 2020-04-10T07:08:29.135 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_instance=cluster2 2020-04-10T07:08:29.135 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ + shift 2020-04-10T07:08:29.135 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:08:29.136 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:grep -i state+ 2020-04-10T07:08:29.136 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:local instance=cluster2 2020-04-10T07:08:29.136 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local loop_instance 2020-04-10T07:08:29.136 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0 2020-04-10T07:08:29.138 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ cluster2 != cluster2 ] 2020-04-10T07:08:29.138 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:08:29.138 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster2:0 rbd mirror status mirror/test 2020-04-10T07:08:29.138 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2:0 2020-04-10T07:08:29.138 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:08:29.139 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:29.139 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:0 cluster instance 2020-04-10T07:08:29.139 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:0 2020-04-10T07:08:29.139 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:08:29.139 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:29.139 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:08:29.139 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:08:29.139 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster2:0 ] 2020-04-10T07:08:29.140 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster2 2020-04-10T07:08:29.140 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:08:29.140 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:08:29.140 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:08:29.140 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster2:0 cluster2 2020-04-10T07:08:29.140 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster2:0 2020-04-10T07:08:29.140 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:08:29.141 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:29.141 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:0 local_cluster instance 2020-04-10T07:08:29.141 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:0 2020-04-10T07:08:29.141 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:08:29.141 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:29.141 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:08:29.141 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:08:29.142 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster2:0 ] 2020-04-10T07:08:29.142 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster2 2020-04-10T07:08:29.143 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster2 2020-04-10T07:08:29.143 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:08:29.143 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:08:29.143 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster2 --name client.mirror.0 admin socket 2020-04-10T07:08:29.158 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster2-client.mirror.0.asok 2020-04-10T07:08:29.158 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster2-client.mirror.0.asok 2020-04-10T07:08:29.158 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster2-client.mirror.0.asok 2020-04-10T07:08:29.159 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster2-client.mirror.0.asok rbd mirror status mirror/test 2020-04-10T07:08:29.252 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:no valid command found; 1 closest matches: 2020-04-10T07:08:29.253 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status mirror/test1 2020-04-10T07:08:29.253 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:08:29.255 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster2:1 rbd mirror status mirror/test 2020-04-10T07:08:29.255 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2:1 2020-04-10T07:08:29.255 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:08:29.255 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:29.255 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:1 cluster instance 2020-04-10T07:08:29.255 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:1 2020-04-10T07:08:29.255 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:08:29.256 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:29.256 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:08:29.256 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:08:29.256 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster2:1 ] 2020-04-10T07:08:29.256 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster2 2020-04-10T07:08:29.256 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:08:29.256 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:08:29.257 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:08:29.257 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster2:1 cluster2 2020-04-10T07:08:29.257 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster2:1 2020-04-10T07:08:29.257 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:08:29.257 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:29.257 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:1 local_cluster instance 2020-04-10T07:08:29.257 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:1 2020-04-10T07:08:29.258 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:08:29.258 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:29.258 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:08:29.258 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:08:29.258 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster2:1 ] 2020-04-10T07:08:29.258 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster2 2020-04-10T07:08:29.258 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster2 2020-04-10T07:08:29.258 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:08:29.259 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:08:29.259 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster2 --name client.mirror.1 admin socket 2020-04-10T07:08:29.268 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster2-client.mirror.1.asok 2020-04-10T07:08:29.268 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster2-client.mirror.1.asok 2020-04-10T07:08:29.268 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster2-client.mirror.1.asok 2020-04-10T07:08:29.268 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster2-client.mirror.1.asok rbd mirror status mirror/test 2020-04-10T07:08:29.366 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:08:29.367 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ status_result= "state": "Stopped" 2020-04-10T07:08:29.367 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo "state": "Stopped" 2020-04-10T07:08:29.367 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep -i Replaying 2020-04-10T07:08:29.369 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test started = stopped 2020-04-10T07:08:29.369 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 2 2020-04-10T07:08:31.370 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_image_replay_state cluster2 mirror test started 2020-04-10T07:08:31.370 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:08:31.370 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:31.370 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:31.370 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local test_state=started 2020-04-10T07:08:31.370 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_result 2020-04-10T07:08:31.371 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_state=stopped 2020-04-10T07:08:31.372 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemons cluster2 rbd mirror status mirror/test 2020-04-10T07:08:31.372 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_instance=cluster2 2020-04-10T07:08:31.372 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:08:31.372 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:08:31.372 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance=cluster2 2020-04-10T07:08:31.372 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ + local loop_instance 2020-04-10T07:08:31.373 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0 2020-04-10T07:08:31.373 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:grep -i state 2020-04-10T07:08:31.373 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ cluster2 != cluster2 ] 2020-04-10T07:08:31.373 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:08:31.375 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster2:0 rbd mirror status mirror/test 2020-04-10T07:08:31.375 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2:0 2020-04-10T07:08:31.375 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:08:31.375 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:31.375 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:0 cluster instance 2020-04-10T07:08:31.375 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:0 2020-04-10T07:08:31.375 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:08:31.376 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:31.376 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:08:31.376 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:08:31.376 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster2:0 ] 2020-04-10T07:08:31.376 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster2 2020-04-10T07:08:31.376 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:08:31.376 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:08:31.377 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:08:31.377 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster2:0 cluster2 2020-04-10T07:08:31.377 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster2:0 2020-04-10T07:08:31.377 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:08:31.377 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:31.377 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:0 local_cluster instance 2020-04-10T07:08:31.377 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:0 2020-04-10T07:08:31.377 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:08:31.378 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:31.378 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:08:31.378 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:08:31.379 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster2:0 ] 2020-04-10T07:08:31.379 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster2 2020-04-10T07:08:31.379 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster2 2020-04-10T07:08:31.379 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:08:31.379 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:08:31.379 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster2 --name client.mirror.0 admin socket 2020-04-10T07:08:31.394 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster2-client.mirror.0.asok 2020-04-10T07:08:31.395 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster2-client.mirror.0.asok 2020-04-10T07:08:31.395 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster2-client.mirror.0.asok 2020-04-10T07:08:31.395 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster2-client.mirror.0.asok rbd mirror status mirror/test 2020-04-10T07:08:31.491 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:no valid command found; 1 closest matches: 2020-04-10T07:08:31.491 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status mirror/test1 2020-04-10T07:08:31.492 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:08:31.495 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster2:1 rbd mirror status mirror/test 2020-04-10T07:08:31.495 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2:1 2020-04-10T07:08:31.495 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:08:31.495 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:31.495 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:1 cluster instance 2020-04-10T07:08:31.495 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:1 2020-04-10T07:08:31.496 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:08:31.496 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:31.496 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:08:31.496 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:08:31.496 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster2:1 ] 2020-04-10T07:08:31.496 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster2 2020-04-10T07:08:31.497 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:08:31.497 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:08:31.497 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:08:31.497 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster2:1 cluster2 2020-04-10T07:08:31.497 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster2:1 2020-04-10T07:08:31.497 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:08:31.498 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:31.498 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:1 local_cluster instance 2020-04-10T07:08:31.498 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:1 2020-04-10T07:08:31.498 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:08:31.498 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:31.498 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:08:31.498 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:08:31.499 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster2:1 ] 2020-04-10T07:08:31.499 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster2 2020-04-10T07:08:31.499 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster2 2020-04-10T07:08:31.499 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:08:31.499 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:08:31.499 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster2 --name client.mirror.1 admin socket 2020-04-10T07:08:31.515 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster2-client.mirror.1.asok 2020-04-10T07:08:31.515 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster2-client.mirror.1.asok 2020-04-10T07:08:31.515 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster2-client.mirror.1.asok 2020-04-10T07:08:31.515 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster2-client.mirror.1.asok rbd mirror status mirror/test 2020-04-10T07:08:31.613 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:08:31.614 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ status_result= "state": "Replaying" 2020-04-10T07:08:31.614 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ + grep -i Replaying 2020-04-10T07:08:31.614 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:echo "state": "Replaying" 2020-04-10T07:08:31.615 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "state": "Replaying" 2020-04-10T07:08:31.615 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ current_state=started 2020-04-10T07:08:31.616 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test started = started 2020-04-10T07:08:31.616 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:08:31.616 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ demote_image cluster1 mirror test 2020-04-10T07:08:31.616 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:31.616 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:31.616 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:31.617 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/test 2020-04-10T07:08:32.139 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Image demoted to non-primary 2020-04-10T07:08:32.146 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_stopped cluster2 mirror test 2020-04-10T07:08:32.146 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:08:32.146 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:32.146 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:32.146 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_state cluster2 mirror test stopped 2020-04-10T07:08:32.146 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:08:32.146 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:32.147 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:32.147 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state=stopped 2020-04-10T07:08:32.147 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local s 2020-04-10T07:08:32.147 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:08:33.147 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_image_replay_state cluster2 mirror test stopped 2020-04-10T07:08:33.148 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:08:33.148 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:33.148 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:33.148 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local test_state=stopped 2020-04-10T07:08:33.148 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_result 2020-04-10T07:08:33.148 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_state=stopped 2020-04-10T07:08:33.149 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemons cluster2 rbd mirror status mirror/test 2020-04-10T07:08:33.149 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_instance=cluster2 2020-04-10T07:08:33.149 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:08:33.149 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:08:33.149 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance=cluster2 2020-04-10T07:08:33.149 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local loop_instance 2020-04-10T07:08:33.149 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ + sleep 0 2020-04-10T07:08:33.150 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:grep -i state 2020-04-10T07:08:33.153 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ cluster2 != cluster2 ] 2020-04-10T07:08:33.153 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:08:33.154 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster2:0 rbd mirror status mirror/test 2020-04-10T07:08:33.154 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2:0 2020-04-10T07:08:33.154 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:08:33.155 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:33.155 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:0 cluster instance 2020-04-10T07:08:33.155 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:0 2020-04-10T07:08:33.156 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:08:33.156 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:33.156 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:08:33.156 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:08:33.156 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster2:0 ] 2020-04-10T07:08:33.156 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster2 2020-04-10T07:08:33.156 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:08:33.157 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:08:33.157 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:08:33.157 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster2:0 cluster2 2020-04-10T07:08:33.157 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster2:0 2020-04-10T07:08:33.157 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:08:33.157 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:33.158 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:0 local_cluster instance 2020-04-10T07:08:33.158 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:0 2020-04-10T07:08:33.158 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:08:33.158 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:33.158 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:08:33.158 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:08:33.158 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster2:0 ] 2020-04-10T07:08:33.158 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster2 2020-04-10T07:08:33.159 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster2 2020-04-10T07:08:33.159 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:08:33.159 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:08:33.159 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster2 --name client.mirror.0 admin socket 2020-04-10T07:08:33.169 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster2-client.mirror.0.asok 2020-04-10T07:08:33.169 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster2-client.mirror.0.asok 2020-04-10T07:08:33.169 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster2-client.mirror.0.asok 2020-04-10T07:08:33.169 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster2-client.mirror.0.asok rbd mirror status mirror/test 2020-04-10T07:08:33.254 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:no valid command found; 1 closest matches: 2020-04-10T07:08:33.254 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status mirror/test1 2020-04-10T07:08:33.255 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:08:33.257 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster2:1 rbd mirror status mirror/test 2020-04-10T07:08:33.257 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2:1 2020-04-10T07:08:33.257 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:08:33.257 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:33.257 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:1 cluster instance 2020-04-10T07:08:33.258 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:1 2020-04-10T07:08:33.258 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:08:33.258 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:33.258 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:08:33.258 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:08:33.258 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster2:1 ] 2020-04-10T07:08:33.258 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster2 2020-04-10T07:08:33.259 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:08:33.259 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:08:33.259 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:08:33.259 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster2:1 cluster2 2020-04-10T07:08:33.259 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster2:1 2020-04-10T07:08:33.259 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:08:33.259 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:33.259 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:1 local_cluster instance 2020-04-10T07:08:33.260 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:1 2020-04-10T07:08:33.260 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:08:33.260 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:33.260 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:08:33.260 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:08:33.260 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster2:1 ] 2020-04-10T07:08:33.260 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster2 2020-04-10T07:08:33.261 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster2 2020-04-10T07:08:33.261 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:08:33.261 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:08:33.261 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster2 --name client.mirror.1 admin socket 2020-04-10T07:08:33.278 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster2-client.mirror.1.asok 2020-04-10T07:08:33.278 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster2-client.mirror.1.asok 2020-04-10T07:08:33.278 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster2-client.mirror.1.asok 2020-04-10T07:08:33.278 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster2-client.mirror.1.asok rbd mirror status mirror/test 2020-04-10T07:08:33.370 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:08:33.371 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ status_result= "state": "Stopped" 2020-04-10T07:08:33.371 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo "state": "Stopped" 2020-04-10T07:08:33.371 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep -i Replaying 2020-04-10T07:08:33.374 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test stopped = stopped 2020-04-10T07:08:33.374 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:08:33.374 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+unknown 2020-04-10T07:08:33.374 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:33.374 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:33.375 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:33.375 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+unknown 2020-04-10T07:08:33.375 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:08:33.375 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:08:33.375 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:08:34.375 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_status_in_pool_dir cluster1 mirror test up+unknown 2020-04-10T07:08:34.375 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:34.375 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:34.375 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:34.375 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+unknown 2020-04-10T07:08:34.375 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:08:34.375 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:08:34.376 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror-test.mirror_status 2020-04-10T07:08:34.376 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror-test.mirror_status 2020-04-10T07:08:34.378 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:08:34.380 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:08:34.380 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test 2020-04-10T07:08:34.380 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ tee /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:08:34.447 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:test: 2020-04-10T07:08:34.447 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: global_id: a9af7d9e-1404-4730-9e18-c0418a915346 2020-04-10T07:08:34.448 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+stopped 2020-04-10T07:08:34.448 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: local image is primary 2020-04-10T07:08:34.448 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: service: mirror.2 on smithi204 2020-04-10T07:08:34.448 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:08:29 2020-04-10T07:08:34.448 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: peer_sites: 2020-04-10T07:08:34.448 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: name: cluster2-DNE 2020-04-10T07:08:34.448 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+error 2020-04-10T07:08:34.449 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: failed to unlink local peer from remote image 2020-04-10T07:08:34.449 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:08:33 2020-04-10T07:08:34.456 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ state: .*up+unknown /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:08:34.458 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 1 2020-04-10T07:08:34.458 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 2 2020-04-10T07:08:36.460 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_status_in_pool_dir cluster1 mirror test up+unknown 2020-04-10T07:08:36.460 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:36.460 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:36.460 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:36.460 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+unknown 2020-04-10T07:08:36.460 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:08:36.461 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:08:36.461 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror-test.mirror_status 2020-04-10T07:08:36.461 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror-test.mirror_status 2020-04-10T07:08:36.461 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:08:36.462 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:08:36.463 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status+ mirror/test 2020-04-10T07:08:36.463 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:tee /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:08:36.536 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:test: 2020-04-10T07:08:36.536 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: global_id: a9af7d9e-1404-4730-9e18-c0418a915346 2020-04-10T07:08:36.537 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+unknown 2020-04-10T07:08:36.537 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: remote image demoted 2020-04-10T07:08:36.537 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: service: mirror.2 on smithi204 2020-04-10T07:08:36.537 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:08:34 2020-04-10T07:08:36.537 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: peer_sites: 2020-04-10T07:08:36.537 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: name: cluster2-DNE 2020-04-10T07:08:36.538 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+unknown 2020-04-10T07:08:36.538 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: remote image demoted 2020-04-10T07:08:36.538 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:08:36 2020-04-10T07:08:36.545 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ state: .*up+unknown /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:08:36.550 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+unknown 2020-04-10T07:08:36.550 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: remote image demoted 2020-04-10T07:08:36.550 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+unknown 2020-04-10T07:08:36.551 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ description: .* /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:08:36.551 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:08:36.551 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+unknown 2020-04-10T07:08:36.551 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^up+ 2020-04-10T07:08:36.551 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep service: *mirror..* on /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:08:36.552 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: service: mirror.2 on smithi204 2020-04-10T07:08:36.552 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:08:36.554 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update=2020-04-10 07:08:34 2020-04-10T07:08:36.554 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+unknown 2020-04-10 07:08:34 2020-04-10T07:08:36.554 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:36.555 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:36.555 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:36.555 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+unknown 2020-04-10T07:08:36.555 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local prev_last_update=2020-04-10 07:08:34 2020-04-10T07:08:36.555 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror.mirror_status 2020-04-10T07:08:36.556 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror.mirror_status 2020-04-10T07:08:36.556 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:08:36.557 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror.mirror_status 2020-04-10T07:08:36.557 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2020-04-10T07:08:36.599 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: failed to query services: (13) Permission denied 2020-04-10T07:08:36.652 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update state 2020-04-10T07:08:36.653 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2020-04-10T07:08:36.658 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ last_update=2020-04-10 07:08:34 2020-04-10T07:08:36.658 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2020-04-10T07:08:36.662 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ state=up+unknown 2020-04-10T07:08:36.663 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+unknown 2020-04-10T07:08:36.663 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep up+unknown 2020-04-10T07:08:36.665 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+unknown 2020-04-10T07:08:36.665 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:08:36.665 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:08:36.665 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_status_in_pool_dir cluster2 mirror test up+unknown 2020-04-10T07:08:36.665 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:08:36.665 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:36.666 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:36.666 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+unknown 2020-04-10T07:08:36.666 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:08:36.666 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:08:36.666 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:08:37.667 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_status_in_pool_dir cluster2 mirror test up+unknown 2020-04-10T07:08:37.667 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:08:37.667 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:37.667 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:37.667 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+unknown 2020-04-10T07:08:37.668 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:08:37.668 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:08:37.668 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster2-mirror-test.mirror_status 2020-04-10T07:08:37.668 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2-mirror-test.mirror_status 2020-04-10T07:08:37.668 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:08:37.669 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:08:37.670 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster2 mirror image status mirror/test 2020-04-10T07:08:37.670 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ tee /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:08:37.743 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:test: 2020-04-10T07:08:37.743 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: global_id: a9af7d9e-1404-4730-9e18-c0418a915346 2020-04-10T07:08:37.743 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+unknown 2020-04-10T07:08:37.743 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: remote image demoted 2020-04-10T07:08:37.743 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: service: mirror.1 on smithi204 2020-04-10T07:08:37.744 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:08:36 2020-04-10T07:08:37.744 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: peer_sites: 2020-04-10T07:08:37.744 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: name: cluster1-DNE 2020-04-10T07:08:37.744 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+unknown 2020-04-10T07:08:37.744 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: remote image demoted 2020-04-10T07:08:37.744 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:08:34 2020-04-10T07:08:37.750 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ state: .*up+unknown /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:08:37.761 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+unknown 2020-04-10T07:08:37.761 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: remote image demoted 2020-04-10T07:08:37.762 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+unknown 2020-04-10T07:08:37.762 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: service: mirror.1 on smithi204 2020-04-10T07:08:37.762 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ description: .* /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:08:37.762 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:08:37.762 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+unknown 2020-04-10T07:08:37.762 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^up+ 2020-04-10T07:08:37.763 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep service: *mirror..* on /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:08:37.763 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:08:37.763 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update=2020-04-10 07:08:36 2020-04-10T07:08:37.763 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_mirror_pool_status_verbose cluster2 mirror test up+unknown 2020-04-10 07:08:36 2020-04-10T07:08:37.763 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:08:37.763 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:37.763 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:37.764 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+unknown 2020-04-10T07:08:37.764 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local prev_last_update=2020-04-10 07:08:36 2020-04-10T07:08:37.764 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster2-mirror.mirror_status 2020-04-10T07:08:37.764 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2-mirror.mirror_status 2020-04-10T07:08:37.764 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:08:37.764 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster2-mirror.mirror_status 2020-04-10T07:08:37.765 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2020-04-10T07:08:37.798 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: failed to query services: (13) Permission denied 2020-04-10T07:08:37.837 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update state 2020-04-10T07:08:37.837 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2020-04-10T07:08:37.840 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ last_update=2020-04-10 07:08:36 2020-04-10T07:08:37.840 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2020-04-10T07:08:37.842 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ state=up+unknown 2020-04-10T07:08:37.842 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ + echo up+unknowngrep 2020-04-10T07:08:37.842 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: up+unknown 2020-04-10T07:08:37.843 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+unknown 2020-04-10T07:08:37.843 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:08:37.844 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:08:37.844 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ promote_image cluster2 mirror test 2020-04-10T07:08:37.844 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:08:37.844 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:37.844 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:37.845 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local force= 2020-04-10T07:08:37.845 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/test 2020-04-10T07:08:38.137 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Image promoted to primary 2020-04-10T07:08:38.157 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_started cluster1 mirror test 2020-04-10T07:08:38.157 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:38.158 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:38.158 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:38.158 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2020-04-10T07:08:38.158 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:38.158 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:38.158 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:38.158 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state=started 2020-04-10T07:08:38.159 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local s 2020-04-10T07:08:38.159 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:08:39.146 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_image_replay_state cluster1 mirror test started 2020-04-10T07:08:39.146 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:39.147 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:39.147 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:39.147 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local test_state=started 2020-04-10T07:08:39.147 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_result 2020-04-10T07:08:39.147 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_state=stopped 2020-04-10T07:08:39.149 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2020-04-10T07:08:39.149 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_instance=cluster1 2020-04-10T07:08:39.149 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift+ 2020-04-10T07:08:39.149 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep -i state 2020-04-10T07:08:39.149 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:local cluster=cluster1 2020-04-10T07:08:39.149 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance=cluster1 2020-04-10T07:08:39.149 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local loop_instance 2020-04-10T07:08:39.150 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0 2020-04-10T07:08:39.150 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ cluster1 != cluster1 ] 2020-04-10T07:08:39.150 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:08:39.151 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2020-04-10T07:08:39.151 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:08:39.151 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:08:39.152 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:39.152 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:08:39.152 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:08:39.152 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:08:39.152 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:39.152 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:39.152 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:08:39.153 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:08:39.153 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:08:39.153 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:39.153 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:08:39.153 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:08:39.153 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster1 2020-04-10T07:08:39.155 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:08:39.155 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:39.155 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:39.155 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:08:39.155 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:08:39.155 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:08:39.155 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:39.156 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:39.156 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:08:39.157 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:08:39.157 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:08:39.157 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:08:39.157 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:08:39.157 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:08:39.157 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:08:39.171 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:08:39.171 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:08:39.171 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:08:39.171 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2020-04-10T07:08:39.268 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:no valid command found; 1 closest matches: 2020-04-10T07:08:39.268 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status mirror/test1 2020-04-10T07:08:39.268 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:08:39.270 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2020-04-10T07:08:39.271 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:1 2020-04-10T07:08:39.271 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:08:39.271 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:39.271 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 cluster instance 2020-04-10T07:08:39.271 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:08:39.271 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:08:39.271 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:39.272 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:39.272 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:08:39.272 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:08:39.272 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:08:39.272 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:39.272 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:08:39.272 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:08:39.273 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:1 cluster1 2020-04-10T07:08:39.273 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:1 2020-04-10T07:08:39.273 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:39.273 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:39.273 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2020-04-10T07:08:39.273 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:08:39.273 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:08:39.274 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:39.274 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:39.274 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:08:39.274 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:08:39.274 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:08:39.274 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:08:39.274 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:08:39.274 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:08:39.275 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2020-04-10T07:08:39.290 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:08:39.291 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:08:39.291 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:08:39.291 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2020-04-10T07:08:39.390 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:Invalid command: unused arguments: ['mirror/test'] 2020-04-10T07:08:39.391 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status : get status for rbd mirror 2020-04-10T07:08:39.391 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:08:39.393 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:2 rbd mirror status mirror/test 2020-04-10T07:08:39.394 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:2 2020-04-10T07:08:39.394 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:08:39.394 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:39.394 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 cluster instance 2020-04-10T07:08:39.394 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:08:39.394 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:08:39.394 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:39.395 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:39.395 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:08:39.395 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:08:39.395 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:08:39.395 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:39.395 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:08:39.395 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:08:39.395 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:2 cluster1 2020-04-10T07:08:39.396 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:2 2020-04-10T07:08:39.396 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:39.396 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:39.396 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2020-04-10T07:08:39.396 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:08:39.396 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:08:39.396 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:39.397 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:39.397 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:08:39.397 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:08:39.397 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:08:39.397 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:08:39.397 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:08:39.397 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:08:39.397 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2020-04-10T07:08:39.408 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:08:39.408 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:08:39.408 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:08:39.408 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2020-04-10T07:08:39.501 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:08:39.502 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ status_result= "state": "Replaying" 2020-04-10T07:08:39.502 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ + grep -i Replaying 2020-04-10T07:08:39.502 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:echo "state": "Replaying" 2020-04-10T07:08:39.505 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "state": "Replaying" 2020-04-10T07:08:39.505 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ current_state=started 2020-04-10T07:08:39.505 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test started = started 2020-04-10T07:08:39.505 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:08:39.506 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror test 2020-04-10T07:08:39.506 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1 2020-04-10T07:08:39.506 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:08:39.506 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:39.506 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:39.506 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ snapshot = journal ] 2020-04-10T07:08:39.506 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ snapshot = snapshot ] 2020-04-10T07:08:39.506 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror test 2020-04-10T07:08:39.507 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1 2020-04-10T07:08:39.507 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:08:39.507 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:39.507 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:39.507 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster2-mirror-test.status 2020-04-10T07:08:39.507 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2-mirror-test.status 2020-04-10T07:08:39.507 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:08:39.508 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.status 2020-04-10T07:08:39.508 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror-test.status 2020-04-10T07:08:39.508 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror-test.status 2020-04-10T07:08:39.508 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:08:39.509 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.status 2020-04-10T07:08:39.509 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mirror_image_snapshot cluster2 mirror test 2020-04-10T07:08:39.509 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:08:39.509 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:39.510 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:39.510 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test 2020-04-10T07:08:40.159 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Snapshot ID: 27 2020-04-10T07:08:40.170 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_newest_mirror_snapshot cluster2 mirror test /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.status 2020-04-10T07:08:40.171 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:08:40.171 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:40.171 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:40.171 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local log=/tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.status 2020-04-10T07:08:40.171 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 snap list --all mirror/test --format xml 2020-04-10T07:08:40.171 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2020-04-10T07:08:40.227 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //snapshot/id 2020-04-10T07:08:40.231 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snapshot_id=27 2020-04-10T07:08:40.231 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ true 2020-04-10T07:08:40.231 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0.2 2020-04-10T07:08:40.433 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.status 2020-04-10T07:08:40.433 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:40.433 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:40.434 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:40.434 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.status 2020-04-10T07:08:40.434 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2020-04-10T07:08:40.434 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2020-04-10T07:08:40.502 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2020-04-10T07:08:40.506 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local primary_snapshot_id=26 2020-04-10T07:08:40.507 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test 27 = 26 2020-04-10T07:08:40.507 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0.4 2020-04-10T07:08:40.908 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.status 2020-04-10T07:08:40.909 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:40.909 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:40.909 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:40.909 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.status 2020-04-10T07:08:40.909 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2020-04-10T07:08:40.909 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2020-04-10T07:08:40.976 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2020-04-10T07:08:40.982 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local primary_snapshot_id=26 2020-04-10T07:08:40.982 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test 27 = 26 2020-04-10T07:08:40.982 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0.8 2020-04-10T07:08:41.784 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.status 2020-04-10T07:08:41.784 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:41.784 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:41.784 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:41.784 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.status 2020-04-10T07:08:41.785 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2020-04-10T07:08:41.785 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2020-04-10T07:08:41.856 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2020-04-10T07:08:41.861 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local primary_snapshot_id=27 2020-04-10T07:08:41.862 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test 27 = 27 2020-04-10T07:08:41.862 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:08:41.862 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2020-04-10T07:08:41.862 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:41.862 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:41.862 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:41.862 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:08:41.863 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:08:41.863 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:08:41.863 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:08:42.863 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying 2020-04-10T07:08:42.863 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:42.863 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:42.863 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:42.864 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:08:42.864 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:08:42.864 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:08:42.864 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror-test.mirror_status 2020-04-10T07:08:42.864 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror-test.mirror_status 2020-04-10T07:08:42.864 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:08:42.866 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:08:42.866 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test 2020-04-10T07:08:42.866 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ tee /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:08:42.943 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:test: 2020-04-10T07:08:42.944 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: global_id: a9af7d9e-1404-4730-9e18-c0418a915346 2020-04-10T07:08:42.944 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+replaying 2020-04-10T07:08:42.944 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: replaying, {"replay_state": "idle", "remote_snapshot_timestamp": 1586502518} 2020-04-10T07:08:42.944 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: service: mirror.2 on smithi204 2020-04-10T07:08:42.944 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:08:39 2020-04-10T07:08:42.944 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: peer_sites: 2020-04-10T07:08:42.944 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: name: cluster2-DNE 2020-04-10T07:08:42.945 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+stopped 2020-04-10T07:08:42.945 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: local image is primary 2020-04-10T07:08:42.945 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:08:41 2020-04-10T07:08:42.952 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ state: .*up+replaying /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:08:42.954 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+replaying 2020-04-10T07:08:42.954 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ description: .* /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:08:42.956 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: replaying, {"replay_state": "idle", "remote_snapshot_timestamp": 1586502518} 2020-04-10T07:08:42.956 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:08:42.957 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+replaying 2020-04-10T07:08:42.957 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^up+ 2020-04-10T07:08:42.957 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+replaying 2020-04-10T07:08:42.958 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep service: *mirror..* on /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:08:42.959 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: service: mirror.2 on smithi204 2020-04-10T07:08:42.960 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:08:42.962 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update=2020-04-10 07:08:39 2020-04-10T07:08:42.962 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying 2020-04-10 07:08:39 2020-04-10T07:08:42.962 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:42.962 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:42.963 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:42.963 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:08:42.963 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local prev_last_update=2020-04-10 07:08:39 2020-04-10T07:08:42.963 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror.mirror_status 2020-04-10T07:08:42.963 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror.mirror_status 2020-04-10T07:08:42.963 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:08:42.964 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror.mirror_status 2020-04-10T07:08:42.964 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2020-04-10T07:08:43.009 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: failed to query services: (13) Permission denied 2020-04-10T07:08:43.065 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update state 2020-04-10T07:08:43.066 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2020-04-10T07:08:43.066 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ last_update=2020-04-10 07:08:39 2020-04-10T07:08:43.066 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2020-04-10T07:08:43.070 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ state=up+replaying 2020-04-10T07:08:43.071 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+replaying 2020-04-10T07:08:43.071 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep up+replaying 2020-04-10T07:08:43.072 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+replaying 2020-04-10T07:08:43.073 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:08:43.073 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:08:43.073 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_status_in_pool_dir cluster2 mirror test up+stopped 2020-04-10T07:08:43.073 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:08:43.073 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:43.073 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:43.073 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+stopped 2020-04-10T07:08:43.074 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:08:43.074 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:08:43.074 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:08:44.074 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_status_in_pool_dir cluster2 mirror test up+stopped 2020-04-10T07:08:44.074 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:08:44.074 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:44.074 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:44.074 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+stopped 2020-04-10T07:08:44.075 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:08:44.075 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:08:44.075 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster2-mirror-test.mirror_status 2020-04-10T07:08:44.075 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2-mirror-test.mirror_status 2020-04-10T07:08:44.075 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:08:44.076 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:08:44.076 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster2 mirror image status mirror/test 2020-04-10T07:08:44.077 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ tee /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:08:44.141 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:test: 2020-04-10T07:08:44.141 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: global_id: a9af7d9e-1404-4730-9e18-c0418a915346 2020-04-10T07:08:44.141 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+stopped 2020-04-10T07:08:44.141 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: local image is primary 2020-04-10T07:08:44.141 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: service: mirror.1 on smithi204 2020-04-10T07:08:44.141 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:08:41 2020-04-10T07:08:44.142 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: peer_sites: 2020-04-10T07:08:44.142 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: name: cluster1-DNE 2020-04-10T07:08:44.142 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+replaying 2020-04-10T07:08:44.142 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: replaying, {"replay_state": "idle", "remote_snapshot_timestamp": 1586502518} 2020-04-10T07:08:44.142 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:08:39 2020-04-10T07:08:44.142 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: snapshots: 2020-04-10T07:08:44.142 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: 17 .mirror.primary.a9af7d9e-1404-4730-9e18-c0418a915346.ae5a942a-32bb-40d8-b4b6-4b6da2b8780c (peer_uuids:[90b6cfa2-5481-4edf-b462-5494aab62f6f]) 2020-04-10T07:08:44.142 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: 21 .mirror.primary.a9af7d9e-1404-4730-9e18-c0418a915346.720d033f-30db-4fdf-9736-fd21d1dfaf57 (peer_uuids:[90b6cfa2-5481-4edf-b462-5494aab62f6f]) 2020-04-10T07:08:44.143 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: 23 .mirror.primary.a9af7d9e-1404-4730-9e18-c0418a915346.89218af1-114d-4250-8a2f-0710ac21a25a (demoted peer_uuids:[90b6cfa2-5481-4edf-b462-5494aab62f6f]) 2020-04-10T07:08:44.143 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: 27 .mirror.primary.a9af7d9e-1404-4730-9e18-c0418a915346.56964f43-33bc-42bb-89f8-032b8753f629 (peer_uuids:[90b6cfa2-5481-4edf-b462-5494aab62f6f]) 2020-04-10T07:08:44.151 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ state: .*up+stopped /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:08:44.152 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+stopped 2020-04-10T07:08:44.152 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ description: .* /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:08:44.153 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: local image is primary 2020-04-10T07:08:44.153 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:08:44.154 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+stopped 2020-04-10T07:08:44.154 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^up+ 2020-04-10T07:08:44.154 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep service: *mirror..* on /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:08:44.155 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+stopped 2020-04-10T07:08:44.155 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: service: mirror.1 on smithi204 2020-04-10T07:08:44.156 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:08:44.158 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update=2020-04-10 07:08:41 2020-04-10T07:08:44.158 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_mirror_pool_status_verbose cluster2 mirror test up+stopped 2020-04-10 07:08:41 2020-04-10T07:08:44.158 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:08:44.158 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:44.158 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:44.159 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+stopped 2020-04-10T07:08:44.159 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local prev_last_update=2020-04-10 07:08:41 2020-04-10T07:08:44.159 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster2-mirror.mirror_status 2020-04-10T07:08:44.159 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2-mirror.mirror_status 2020-04-10T07:08:44.159 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:08:44.160 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster2-mirror.mirror_status 2020-04-10T07:08:44.160 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2020-04-10T07:08:44.193 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: failed to query services: (13) Permission denied 2020-04-10T07:08:44.228 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update state 2020-04-10T07:08:44.229 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2020-04-10T07:08:44.231 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ last_update=2020-04-10 07:08:41 2020-04-10T07:08:44.231 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2020-04-10T07:08:44.233 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ state=up+stopped 2020-04-10T07:08:44.233 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+stopped 2020-04-10T07:08:44.234 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep up+stopped 2020-04-10T07:08:44.236 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+stopped 2020-04-10T07:08:44.236 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:08:44.236 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:08:44.236 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ compare_images mirror test 2020-04-10T07:08:44.236 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:44.236 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:44.237 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local ret=0 2020-04-10T07:08:44.237 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster2-mirror-test.export 2020-04-10T07:08:44.237 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2-mirror-test.export 2020-04-10T07:08:44.237 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:08:44.237 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local rmt_export=/tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.export 2020-04-10T07:08:44.237 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror-test.export 2020-04-10T07:08:44.238 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror-test.export 2020-04-10T07:08:44.238 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:08:44.238 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local loc_export=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.export 2020-04-10T07:08:44.239 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rm -f /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.export /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.export 2020-04-10T07:08:44.239 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 export mirror/test /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.export 2020-04-10T07:08:44.420 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2020-04-10T07:08:44.424 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 export mirror/test /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.export 2020-04-10T07:08:44.617 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2020-04-10T07:08:44.621 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cmp /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.export /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.export 2020-04-10T07:08:44.693 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rm -f /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.export /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.export 2020-04-10T07:08:44.732 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:08:44.733 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ demote_image cluster2 mirror test 2020-04-10T07:08:44.733 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:08:44.733 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:44.733 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:44.733 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster2 mirror image demote mirror/test 2020-04-10T07:08:45.163 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Image demoted to non-primary 2020-04-10T07:08:45.169 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_stopped cluster1 mirror test 2020-04-10T07:08:45.169 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:45.169 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:45.170 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:45.170 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_state cluster1 mirror test stopped 2020-04-10T07:08:45.170 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:45.170 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:45.170 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:45.170 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state=stopped 2020-04-10T07:08:45.171 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local s 2020-04-10T07:08:45.171 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:08:46.171 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_image_replay_state cluster1 mirror test stopped 2020-04-10T07:08:46.171 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:46.171 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:46.171 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:46.171 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local test_state=stopped 2020-04-10T07:08:46.172 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_result 2020-04-10T07:08:46.172 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_state=stopped 2020-04-10T07:08:46.172 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2020-04-10T07:08:46.172 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_instance=cluster1 2020-04-10T07:08:46.172 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:08:46.172 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:46.172 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance=cluster1 2020-04-10T07:08:46.173 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local loop_instance 2020-04-10T07:08:46.173 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ + sleep 0 2020-04-10T07:08:46.173 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:grep -i state 2020-04-10T07:08:46.175 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ cluster1 != cluster1 ] 2020-04-10T07:08:46.175 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:08:46.175 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2020-04-10T07:08:46.175 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:08:46.175 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:08:46.176 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:46.176 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:08:46.176 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:08:46.176 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:08:46.176 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:46.176 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:46.176 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:08:46.177 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:08:46.177 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:08:46.177 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:46.177 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:08:46.177 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:08:46.177 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster1 2020-04-10T07:08:46.177 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:08:46.177 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:46.178 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:46.178 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:08:46.178 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:08:46.178 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:08:46.178 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:46.178 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:46.178 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:08:46.178 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:08:46.179 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:08:46.179 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:08:46.179 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:08:46.180 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:08:46.180 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:08:46.190 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:08:46.190 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:08:46.190 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:08:46.190 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2020-04-10T07:08:46.283 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:no valid command found; 1 closest matches: 2020-04-10T07:08:46.283 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status mirror/test1 2020-04-10T07:08:46.283 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:08:46.287 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2020-04-10T07:08:46.288 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:1 2020-04-10T07:08:46.288 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:08:46.288 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:46.288 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 cluster instance 2020-04-10T07:08:46.288 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:08:46.288 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:08:46.288 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:46.289 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:46.289 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:08:46.289 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:08:46.289 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:08:46.289 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:46.289 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:08:46.289 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:08:46.289 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:1 cluster1 2020-04-10T07:08:46.290 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:1 2020-04-10T07:08:46.290 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:46.290 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:46.290 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2020-04-10T07:08:46.290 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:08:46.290 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:08:46.290 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:46.291 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:46.291 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:08:46.291 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:08:46.291 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:08:46.291 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:08:46.291 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:08:46.291 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:08:46.291 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2020-04-10T07:08:46.308 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:08:46.308 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:08:46.308 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:08:46.308 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2020-04-10T07:08:46.395 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:Invalid command: unused arguments: ['mirror/test'] 2020-04-10T07:08:46.395 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status : get status for rbd mirror 2020-04-10T07:08:46.396 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:08:46.398 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:2 rbd mirror status mirror/test 2020-04-10T07:08:46.398 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:2 2020-04-10T07:08:46.398 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:08:46.398 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:46.398 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 cluster instance 2020-04-10T07:08:46.398 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:08:46.398 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:08:46.399 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:46.399 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:46.399 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:08:46.399 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:08:46.399 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:08:46.399 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:46.399 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:08:46.400 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:08:46.400 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:2 cluster1 2020-04-10T07:08:46.400 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:2 2020-04-10T07:08:46.400 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:46.400 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:46.400 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2020-04-10T07:08:46.400 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:08:46.401 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:08:46.401 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:46.401 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:08:46.401 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:08:46.401 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:08:46.401 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:08:46.401 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:08:46.401 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:08:46.402 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:08:46.402 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2020-04-10T07:08:46.416 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:08:46.416 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:08:46.416 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:08:46.416 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2020-04-10T07:08:46.513 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:08:46.513 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ status_result= "state": "Stopped" 2020-04-10T07:08:46.513 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ + grep -i Replaying 2020-04-10T07:08:46.514 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:echo "state": "Stopped" 2020-04-10T07:08:46.515 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test stopped = stopped 2020-04-10T07:08:46.515 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:08:46.515 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+unknown 2020-04-10T07:08:46.515 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:46.515 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:46.516 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:46.516 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+unknown 2020-04-10T07:08:46.516 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:08:46.516 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:08:46.516 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:08:47.516 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_status_in_pool_dir cluster1 mirror test up+unknown 2020-04-10T07:08:47.517 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:47.517 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:47.517 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:47.517 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+unknown 2020-04-10T07:08:47.517 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:08:47.517 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:08:47.518 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror-test.mirror_status 2020-04-10T07:08:47.518 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror-test.mirror_status 2020-04-10T07:08:47.518 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:08:47.519 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:08:47.519 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test 2020-04-10T07:08:47.519 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ tee /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:08:47.599 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:test: 2020-04-10T07:08:47.600 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: global_id: a9af7d9e-1404-4730-9e18-c0418a915346 2020-04-10T07:08:47.600 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+error 2020-04-10T07:08:47.600 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: failed to unlink local peer from remote image 2020-04-10T07:08:47.600 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: service: mirror.2 on smithi204 2020-04-10T07:08:47.600 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:08:46 2020-04-10T07:08:47.600 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: peer_sites: 2020-04-10T07:08:47.601 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: name: cluster2-DNE 2020-04-10T07:08:47.601 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+unknown 2020-04-10T07:08:47.601 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: remote image demoted 2020-04-10T07:08:47.601 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:08:46 2020-04-10T07:08:47.607 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ state: .*up+unknown /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:08:47.609 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 1 2020-04-10T07:08:47.610 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 2 2020-04-10T07:08:49.611 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_status_in_pool_dir cluster1 mirror test up+unknown 2020-04-10T07:08:49.611 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:49.612 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:49.612 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:49.612 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+unknown 2020-04-10T07:08:49.612 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:08:49.612 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:08:49.612 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror-test.mirror_status 2020-04-10T07:08:49.613 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror-test.mirror_status 2020-04-10T07:08:49.613 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:08:49.614 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:08:49.614 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test 2020-04-10T07:08:49.614 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ tee /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:08:49.695 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:test: 2020-04-10T07:08:49.695 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: global_id: a9af7d9e-1404-4730-9e18-c0418a915346 2020-04-10T07:08:49.695 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+unknown 2020-04-10T07:08:49.696 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: remote image demoted 2020-04-10T07:08:49.696 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: service: mirror.2 on smithi204 2020-04-10T07:08:49.696 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:08:49 2020-04-10T07:08:49.696 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: peer_sites: 2020-04-10T07:08:49.696 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: name: cluster2-DNE 2020-04-10T07:08:49.696 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+unknown 2020-04-10T07:08:49.696 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: remote image demoted 2020-04-10T07:08:49.697 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:08:46 2020-04-10T07:08:49.704 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ state: .*up+unknown /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:08:49.706 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+unknown 2020-04-10T07:08:49.706 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ description: .* /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:08:49.709 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: remote image demoted 2020-04-10T07:08:49.709 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+unknown 2020-04-10T07:08:49.709 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:08:49.709 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+unknown 2020-04-10T07:08:49.709 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^up+ 2020-04-10T07:08:49.709 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep service: *mirror..* on /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:08:49.710 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: service: mirror.2 on smithi204 2020-04-10T07:08:49.711 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:08:49.713 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update=2020-04-10 07:08:49 2020-04-10T07:08:49.714 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+unknown 2020-04-10 07:08:49 2020-04-10T07:08:49.714 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:49.714 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:49.714 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:49.714 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+unknown 2020-04-10T07:08:49.714 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local prev_last_update=2020-04-10 07:08:49 2020-04-10T07:08:49.714 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror.mirror_status 2020-04-10T07:08:49.715 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror.mirror_status 2020-04-10T07:08:49.715 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:08:49.715 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror.mirror_status 2020-04-10T07:08:49.715 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2020-04-10T07:08:49.757 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: failed to query services: (13) Permission denied 2020-04-10T07:08:49.807 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update state 2020-04-10T07:08:49.807 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2020-04-10T07:08:49.812 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ last_update=2020-04-10 07:08:49 2020-04-10T07:08:49.812 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2020-04-10T07:08:49.818 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ state=up+unknown 2020-04-10T07:08:49.818 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+unknown 2020-04-10T07:08:49.818 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep up+unknown 2020-04-10T07:08:49.819 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+unknown 2020-04-10T07:08:49.820 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:08:49.820 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:08:49.820 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_status_in_pool_dir cluster2 mirror test up+unknown 2020-04-10T07:08:49.820 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:08:49.821 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:49.821 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:49.821 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+unknown 2020-04-10T07:08:49.821 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:08:49.821 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:08:49.821 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:08:50.821 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_status_in_pool_dir cluster2 mirror test up+unknown 2020-04-10T07:08:50.821 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:08:50.821 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:50.822 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:50.822 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+unknown 2020-04-10T07:08:50.822 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:08:50.822 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:08:50.822 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster2-mirror-test.mirror_status 2020-04-10T07:08:50.823 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2-mirror-test.mirror_status 2020-04-10T07:08:50.823 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:08:50.824 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:08:50.824 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster2 mirror image status mirror/test 2020-04-10T07:08:50.824 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ tee /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:08:50.886 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:test: 2020-04-10T07:08:50.887 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: global_id: a9af7d9e-1404-4730-9e18-c0418a915346 2020-04-10T07:08:50.887 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+unknown 2020-04-10T07:08:50.887 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: remote image demoted 2020-04-10T07:08:50.887 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: service: mirror.1 on smithi204 2020-04-10T07:08:50.887 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:08:46 2020-04-10T07:08:50.887 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: peer_sites: 2020-04-10T07:08:50.888 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: name: cluster1-DNE 2020-04-10T07:08:50.888 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+unknown 2020-04-10T07:08:50.888 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: remote image demoted 2020-04-10T07:08:50.888 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:08:49 2020-04-10T07:08:50.892 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ state: .*up+unknown /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:08:50.894 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+unknown 2020-04-10T07:08:50.894 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ description: .* /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:08:50.895 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: remote image demoted 2020-04-10T07:08:50.896 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:08:50.896 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+unknown 2020-04-10T07:08:50.896 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^up+ 2020-04-10T07:08:50.896 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+unknown 2020-04-10T07:08:50.897 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep service: *mirror..* on /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:08:50.904 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: service: mirror.1 on smithi204 2020-04-10T07:08:50.904 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:08:50.905 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update=2020-04-10 07:08:46 2020-04-10T07:08:50.905 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_mirror_pool_status_verbose cluster2 mirror test up+unknown 2020-04-10 07:08:46 2020-04-10T07:08:50.905 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:08:50.905 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:50.905 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:50.905 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+unknown 2020-04-10T07:08:50.905 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local prev_last_update=2020-04-10 07:08:46 2020-04-10T07:08:50.906 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster2-mirror.mirror_status 2020-04-10T07:08:50.906 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2-mirror.mirror_status 2020-04-10T07:08:50.906 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:08:50.906 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster2-mirror.mirror_status 2020-04-10T07:08:50.906 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2020-04-10T07:08:50.945 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: failed to query services: (13) Permission denied 2020-04-10T07:08:50.981 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update state 2020-04-10T07:08:50.981 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2020-04-10T07:08:50.984 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ last_update=2020-04-10 07:08:46 2020-04-10T07:08:50.984 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2020-04-10T07:08:50.987 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ state=up+unknown 2020-04-10T07:08:50.988 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+unknown 2020-04-10T07:08:50.988 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep up+unknown 2020-04-10T07:08:50.988 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:08:50.988 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:08:50.988 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ promote_image cluster1 mirror test 2020-04-10T07:08:50.988 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:50.988 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:50.989 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:50.989 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local force= 2020-04-10T07:08:50.989 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster1 mirror image promote mirror/test 2020-04-10T07:08:50.989 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+unknown 2020-04-10T07:08:52.083 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Image promoted to primary 2020-04-10T07:08:52.096 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_started cluster2 mirror test 2020-04-10T07:08:52.097 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:08:52.097 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:52.097 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:52.097 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_state cluster2 mirror test started 2020-04-10T07:08:52.097 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:08:52.097 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:52.098 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:52.098 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state=started 2020-04-10T07:08:52.098 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local s 2020-04-10T07:08:52.098 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:08:53.097 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_image_replay_state cluster2 mirror test started 2020-04-10T07:08:53.098 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:08:53.098 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:53.098 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:53.098 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local test_state=started 2020-04-10T07:08:53.098 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_result 2020-04-10T07:08:53.098 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_state=stopped 2020-04-10T07:08:53.100 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemons cluster2 rbd mirror status mirror/test 2020-04-10T07:08:53.100 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_instance=cluster2 2020-04-10T07:08:53.100 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:08:53.100 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ + local cluster=cluster2 2020-04-10T07:08:53.100 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:grep -i state 2020-04-10T07:08:53.100 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance=cluster2 2020-04-10T07:08:53.101 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local loop_instance 2020-04-10T07:08:53.101 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0 2020-04-10T07:08:53.101 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ cluster2 != cluster2 ] 2020-04-10T07:08:53.101 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:08:53.102 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster2:0 rbd mirror status mirror/test 2020-04-10T07:08:53.102 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2:0 2020-04-10T07:08:53.103 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:08:53.103 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:53.103 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:0 cluster instance 2020-04-10T07:08:53.103 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:0 2020-04-10T07:08:53.103 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:08:53.103 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:53.103 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:08:53.104 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:08:53.104 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster2:0 ] 2020-04-10T07:08:53.104 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster2 2020-04-10T07:08:53.104 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:08:53.104 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:08:53.104 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:08:53.104 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster2:0 cluster2 2020-04-10T07:08:53.106 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster2:0 2020-04-10T07:08:53.106 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:08:53.106 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:53.106 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:0 local_cluster instance 2020-04-10T07:08:53.106 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:0 2020-04-10T07:08:53.106 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:08:53.113 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:53.113 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:08:53.113 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:08:53.113 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster2:0 ] 2020-04-10T07:08:53.113 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster2 2020-04-10T07:08:53.113 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster2 2020-04-10T07:08:53.114 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:08:53.114 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:08:53.114 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster2 --name client.mirror.0 admin socket 2020-04-10T07:08:53.126 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster2-client.mirror.0.asok 2020-04-10T07:08:53.126 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster2-client.mirror.0.asok 2020-04-10T07:08:53.126 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster2-client.mirror.0.asok 2020-04-10T07:08:53.126 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster2-client.mirror.0.asok rbd mirror status mirror/test 2020-04-10T07:08:53.213 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:no valid command found; 1 closest matches: 2020-04-10T07:08:53.213 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status mirror/test1 2020-04-10T07:08:53.213 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:08:53.215 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster2:1 rbd mirror status mirror/test 2020-04-10T07:08:53.215 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2:1 2020-04-10T07:08:53.216 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:08:53.216 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:53.216 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:1 cluster instance 2020-04-10T07:08:53.216 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:1 2020-04-10T07:08:53.216 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:08:53.216 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:53.216 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:08:53.217 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:08:53.217 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster2:1 ] 2020-04-10T07:08:53.217 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster2 2020-04-10T07:08:53.217 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:08:53.217 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:08:53.217 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:08:53.217 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster2:1 cluster2 2020-04-10T07:08:53.218 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster2:1 2020-04-10T07:08:53.218 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:08:53.218 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:53.218 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:1 local_cluster instance 2020-04-10T07:08:53.218 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:1 2020-04-10T07:08:53.218 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:08:53.218 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:53.218 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:08:53.219 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:08:53.219 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster2:1 ] 2020-04-10T07:08:53.219 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster2 2020-04-10T07:08:53.219 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster2 2020-04-10T07:08:53.219 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:08:53.219 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:08:53.219 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster2 --name client.mirror.1 admin socket 2020-04-10T07:08:53.238 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster2-client.mirror.1.asok 2020-04-10T07:08:53.238 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster2-client.mirror.1.asok 2020-04-10T07:08:53.238 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster2-client.mirror.1.asok 2020-04-10T07:08:53.238 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster2-client.mirror.1.asok rbd mirror status mirror/test 2020-04-10T07:08:53.334 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:08:53.334 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ status_result= "state": "Stopped" 2020-04-10T07:08:53.334 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ + echo "state": "Stopped" 2020-04-10T07:08:53.335 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:grep -i Replaying 2020-04-10T07:08:53.335 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test started = stopped 2020-04-10T07:08:53.335 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 2 2020-04-10T07:08:55.337 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_image_replay_state cluster2 mirror test started 2020-04-10T07:08:55.337 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:08:55.337 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:55.337 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:55.337 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local test_state=started 2020-04-10T07:08:55.337 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_result 2020-04-10T07:08:55.338 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_state=stopped 2020-04-10T07:08:55.339 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemons cluster2 rbd mirror status mirror/test 2020-04-10T07:08:55.339 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_instance=cluster2 2020-04-10T07:08:55.339 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ + shift 2020-04-10T07:08:55.339 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:08:55.339 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:grep+ -ilocal state instance=cluster2 2020-04-10T07:08:55.340 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: 2020-04-10T07:08:55.340 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local loop_instance 2020-04-10T07:08:55.340 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0 2020-04-10T07:08:55.340 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ cluster2 != cluster2 ] 2020-04-10T07:08:55.340 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:08:55.342 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster2:0 rbd mirror status mirror/test 2020-04-10T07:08:55.342 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2:0 2020-04-10T07:08:55.342 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:08:55.343 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:55.343 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:0 cluster instance 2020-04-10T07:08:55.343 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:0 2020-04-10T07:08:55.343 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:08:55.343 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:55.343 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:08:55.343 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:08:55.344 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster2:0 ] 2020-04-10T07:08:55.344 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster2 2020-04-10T07:08:55.344 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:08:55.344 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:08:55.344 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:08:55.344 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster2:0 cluster2 2020-04-10T07:08:55.345 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster2:0 2020-04-10T07:08:55.345 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:08:55.345 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:55.345 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:0 local_cluster instance 2020-04-10T07:08:55.345 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:0 2020-04-10T07:08:55.346 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:08:55.346 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:55.347 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:08:55.347 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:08:55.347 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster2:0 ] 2020-04-10T07:08:55.347 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster2 2020-04-10T07:08:55.347 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster2 2020-04-10T07:08:55.347 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:08:55.347 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:08:55.348 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster2 --name client.mirror.0 admin socket 2020-04-10T07:08:55.363 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster2-client.mirror.0.asok 2020-04-10T07:08:55.364 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster2-client.mirror.0.asok 2020-04-10T07:08:55.364 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster2-client.mirror.0.asok 2020-04-10T07:08:55.364 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster2-client.mirror.0.asok rbd mirror status mirror/test 2020-04-10T07:08:55.457 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:no valid command found; 1 closest matches: 2020-04-10T07:08:55.457 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status mirror/test1 2020-04-10T07:08:55.457 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:08:55.460 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster2:1 rbd mirror status mirror/test 2020-04-10T07:08:55.460 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2:1 2020-04-10T07:08:55.460 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:08:55.460 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:55.460 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:1 cluster instance 2020-04-10T07:08:55.461 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:1 2020-04-10T07:08:55.461 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:08:55.461 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:55.461 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:08:55.462 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:08:55.462 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster2:1 ] 2020-04-10T07:08:55.462 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster2 2020-04-10T07:08:55.462 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:08:55.462 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:08:55.462 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:08:55.463 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster2:1 cluster2 2020-04-10T07:08:55.463 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster2:1 2020-04-10T07:08:55.463 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:08:55.463 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:55.463 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:1 local_cluster instance 2020-04-10T07:08:55.463 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:1 2020-04-10T07:08:55.464 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:08:55.464 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:55.464 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:08:55.464 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:08:55.464 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster2:1 ] 2020-04-10T07:08:55.464 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster2 2020-04-10T07:08:55.465 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster2 2020-04-10T07:08:55.465 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:08:55.465 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:08:55.465 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster2 --name client.mirror.1 admin socket 2020-04-10T07:08:55.480 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster2-client.mirror.1.asok 2020-04-10T07:08:55.480 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster2-client.mirror.1.asok 2020-04-10T07:08:55.481 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster2-client.mirror.1.asok 2020-04-10T07:08:55.481 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster2-client.mirror.1.asok rbd mirror status mirror/test 2020-04-10T07:08:55.580 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:08:55.580 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ status_result= "state": "Stopped" 2020-04-10T07:08:55.580 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo "state": "Stopped" 2020-04-10T07:08:55.580 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep -i Replaying 2020-04-10T07:08:55.581 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test started = stopped 2020-04-10T07:08:55.581 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 4 2020-04-10T07:08:59.582 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_image_replay_state cluster2 mirror test started 2020-04-10T07:08:59.582 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:08:59.583 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:59.583 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:59.583 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local test_state=started 2020-04-10T07:08:59.583 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_result 2020-04-10T07:08:59.583 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_state=stopped 2020-04-10T07:08:59.584 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemons cluster2 rbd mirror status mirror/test 2020-04-10T07:08:59.585 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_instance=cluster2 2020-04-10T07:08:59.585 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift+ 2020-04-10T07:08:59.585 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:08:59.585 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance=cluster2 2020-04-10T07:08:59.585 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local loop_instance 2020-04-10T07:08:59.585 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:grep -i state 2020-04-10T07:08:59.586 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0 2020-04-10T07:08:59.586 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ cluster2 != cluster2 ] 2020-04-10T07:08:59.586 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:08:59.588 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster2:0 rbd mirror status mirror/test 2020-04-10T07:08:59.588 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2:0 2020-04-10T07:08:59.588 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:08:59.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:59.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:0 cluster instance 2020-04-10T07:08:59.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:0 2020-04-10T07:08:59.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:08:59.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:59.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:08:59.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:08:59.590 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster2:0 ] 2020-04-10T07:08:59.590 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster2 2020-04-10T07:08:59.590 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:08:59.590 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:08:59.590 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:08:59.590 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster2:0 cluster2 2020-04-10T07:08:59.591 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster2:0 2020-04-10T07:08:59.591 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:08:59.591 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:59.591 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:0 local_cluster instance 2020-04-10T07:08:59.591 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:0 2020-04-10T07:08:59.591 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:08:59.591 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:59.592 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:08:59.592 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:08:59.592 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster2:0 ] 2020-04-10T07:08:59.592 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster2 2020-04-10T07:08:59.592 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster2 2020-04-10T07:08:59.592 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:08:59.593 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:08:59.593 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster2 --name client.mirror.0 admin socket 2020-04-10T07:08:59.608 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster2-client.mirror.0.asok 2020-04-10T07:08:59.608 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster2-client.mirror.0.asok 2020-04-10T07:08:59.608 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster2-client.mirror.0.asok 2020-04-10T07:08:59.608 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster2-client.mirror.0.asok rbd mirror status mirror/test 2020-04-10T07:08:59.701 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:no valid command found; 1 closest matches: 2020-04-10T07:08:59.702 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status mirror/test1 2020-04-10T07:08:59.702 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:08:59.705 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster2:1 rbd mirror status mirror/test 2020-04-10T07:08:59.705 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2:1 2020-04-10T07:08:59.705 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:08:59.705 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:59.706 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:1 cluster instance 2020-04-10T07:08:59.706 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:1 2020-04-10T07:08:59.706 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:08:59.706 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:59.706 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:08:59.707 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:08:59.707 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster2:1 ] 2020-04-10T07:08:59.707 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster2 2020-04-10T07:08:59.707 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:08:59.707 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:08:59.708 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:08:59.708 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster2:1 cluster2 2020-04-10T07:08:59.708 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster2:1 2020-04-10T07:08:59.708 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:08:59.708 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:08:59.708 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:1 local_cluster instance 2020-04-10T07:08:59.708 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:1 2020-04-10T07:08:59.709 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:08:59.709 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:08:59.709 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:08:59.709 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:08:59.709 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster2:1 ] 2020-04-10T07:08:59.710 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster2 2020-04-10T07:08:59.710 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster2 2020-04-10T07:08:59.710 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:08:59.710 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:08:59.710 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster2 --name client.mirror.1 admin socket 2020-04-10T07:08:59.719 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster2-client.mirror.1.asok 2020-04-10T07:08:59.719 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster2-client.mirror.1.asok 2020-04-10T07:08:59.719 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster2-client.mirror.1.asok 2020-04-10T07:08:59.719 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster2-client.mirror.1.asok rbd mirror status mirror/test 2020-04-10T07:08:59.815 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:08:59.816 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ status_result= "state": "Replaying" 2020-04-10T07:08:59.816 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo "state": "Replaying" 2020-04-10T07:08:59.817 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep -i Replaying 2020-04-10T07:08:59.820 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "state": "Replaying" 2020-04-10T07:08:59.821 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ current_state=started 2020-04-10T07:08:59.821 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test started = started 2020-04-10T07:08:59.821 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:08:59.821 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ write_image cluster1 mirror test 100 2020-04-10T07:08:59.821 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:08:59.822 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:08:59.822 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:08:59.822 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local count=100 2020-04-10T07:08:59.822 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local size= 2020-04-10T07:08:59.822 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 2020-04-10T07:08:59.822 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ size=4096 2020-04-10T07:08:59.822 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 bench mirror/test --io-type write --io-size 4096 --io-threads 1 --io-total 409600 --io-pattern rand 2020-04-10T07:08:59.914 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:bench type write io_size 4096 io_threads 1 bytes 409600 pattern random 2020-04-10T07:08:59.984 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: SEC OPS OPS/SEC BYTES/SEC 2020-04-10T07:08:59.984 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:elapsed: 0 ops: 100 ops/sec: 1470.59 bytes/sec: 5.7 MiB/s 2020-04-10T07:09:00.001 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_replay_complete cluster2 cluster1 mirror test 2020-04-10T07:09:00.001 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster2 2020-04-10T07:09:00.001 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:00.002 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:00.002 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:09:00.002 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ snapshot = journal ] 2020-04-10T07:09:00.002 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ snapshot = snapshot ] 2020-04-10T07:09:00.002 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_snapshot_sync_complete cluster2 cluster1 mirror test 2020-04-10T07:09:00.002 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster2 2020-04-10T07:09:00.002 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:00.003 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:00.003 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:09:00.003 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror-test.status 2020-04-10T07:09:00.003 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror-test.status 2020-04-10T07:09:00.003 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:09:00.004 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.status 2020-04-10T07:09:00.004 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster2-mirror-test.status 2020-04-10T07:09:00.004 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2-mirror-test.status 2020-04-10T07:09:00.005 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:09:00.006 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_status_log=/tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.status 2020-04-10T07:09:00.006 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mirror_image_snapshot cluster1 mirror test 2020-04-10T07:09:00.006 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:00.006 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:00.007 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:09:00.007 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/test 2020-04-10T07:09:01.128 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Snapshot ID: 28 2020-04-10T07:09:01.143 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.status 2020-04-10T07:09:01.143 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:01.143 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:01.143 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:09:01.143 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.status 2020-04-10T07:09:01.143 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2020-04-10T07:09:01.144 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2020-04-10T07:09:01.219 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //snapshot/id 2020-04-10T07:09:01.222 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snapshot_id=28 2020-04-10T07:09:01.222 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ true 2020-04-10T07:09:01.222 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0.2 2020-04-10T07:09:01.424 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_newest_mirror_snapshot cluster2 mirror test /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.status 2020-04-10T07:09:01.425 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:01.425 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:01.425 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:09:01.425 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local log=/tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.status 2020-04-10T07:09:01.425 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 snap list --all mirror/test --format xml 2020-04-10T07:09:01.425 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2020-04-10T07:09:01.486 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2020-04-10T07:09:01.491 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local primary_snapshot_id=27 2020-04-10T07:09:01.491 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test 28 = 27 2020-04-10T07:09:01.491 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0.4 2020-04-10T07:09:01.892 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_newest_mirror_snapshot cluster2 mirror test /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.status 2020-04-10T07:09:01.892 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:01.893 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:01.893 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:09:01.893 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local log=/tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.status 2020-04-10T07:09:01.893 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 snap list --all mirror/test --format xml 2020-04-10T07:09:01.893 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2020-04-10T07:09:01.960 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2020-04-10T07:09:01.964 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local primary_snapshot_id=27 2020-04-10T07:09:01.964 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test 28 = 27 2020-04-10T07:09:01.964 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0.8 2020-04-10T07:09:02.766 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_newest_mirror_snapshot cluster2 mirror test /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.status 2020-04-10T07:09:02.766 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:02.767 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:02.767 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:09:02.767 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local log=/tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.status 2020-04-10T07:09:02.767 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 snap list --all mirror/test --format xml 2020-04-10T07:09:02.767 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2020-04-10T07:09:02.837 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2020-04-10T07:09:02.842 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local primary_snapshot_id=28 2020-04-10T07:09:02.842 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test 28 = 28 2020-04-10T07:09:02.843 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:09:02.843 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+stopped 2020-04-10T07:09:02.843 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:02.843 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:02.843 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:09:02.843 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+stopped 2020-04-10T07:09:02.843 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:09:02.844 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:09:02.844 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:09:03.844 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_status_in_pool_dir cluster1 mirror test up+stopped 2020-04-10T07:09:03.844 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:03.844 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:03.845 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:09:03.845 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+stopped 2020-04-10T07:09:03.845 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:09:03.845 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:09:03.845 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror-test.mirror_status 2020-04-10T07:09:03.845 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror-test.mirror_status 2020-04-10T07:09:03.845 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:09:03.846 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:09:03.847 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test 2020-04-10T07:09:03.847 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ tee /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:09:03.923 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:test: 2020-04-10T07:09:03.924 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: global_id: a9af7d9e-1404-4730-9e18-c0418a915346 2020-04-10T07:09:03.924 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+stopped 2020-04-10T07:09:03.924 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: local image is primary 2020-04-10T07:09:03.924 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: service: mirror.2 on smithi204 2020-04-10T07:09:03.924 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:08:59 2020-04-10T07:09:03.924 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: peer_sites: 2020-04-10T07:09:03.924 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: name: cluster2-DNE 2020-04-10T07:09:03.925 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+replaying 2020-04-10T07:09:03.925 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: replaying, {"replay_state": "idle", "remote_snapshot_timestamp": 1586502532} 2020-04-10T07:09:03.925 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:08:56 2020-04-10T07:09:03.925 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: snapshots: 2020-04-10T07:09:03.925 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: 21 .mirror.primary.a9af7d9e-1404-4730-9e18-c0418a915346.c4826144-e950-45d7-a924-f7e7ebafbbab (peer_uuids:[2571f49c-d767-4112-ae12-d8b71263d58b]) 2020-04-10T07:09:03.925 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: 22 .mirror.primary.a9af7d9e-1404-4730-9e18-c0418a915346.de98fea2-b62b-4ba3-8b00-3a201857b55a (demoted peer_uuids:[2571f49c-d767-4112-ae12-d8b71263d58b]) 2020-04-10T07:09:03.925 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: 28 .mirror.primary.a9af7d9e-1404-4730-9e18-c0418a915346.5e7a1cdb-ecfd-40c7-b31d-bb1a163dec3e (peer_uuids:[2571f49c-d767-4112-ae12-d8b71263d58b]) 2020-04-10T07:09:03.932 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ state: .*up+stopped /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:09:03.934 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+stopped 2020-04-10T07:09:03.934 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ description: .* /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:09:03.936 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: local image is primary 2020-04-10T07:09:03.936 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:09:03.936 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+stopped 2020-04-10T07:09:03.937 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^up+ 2020-04-10T07:09:03.938 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+stopped 2020-04-10T07:09:03.938 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep service: *mirror..* on /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:09:03.939 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: service: mirror.2 on smithi204 2020-04-10T07:09:03.940 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:09:03.942 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update=2020-04-10 07:08:59 2020-04-10T07:09:03.942 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+stopped 2020-04-10 07:08:59 2020-04-10T07:09:03.942 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:03.942 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:03.943 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:09:03.943 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+stopped 2020-04-10T07:09:03.943 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local prev_last_update=2020-04-10 07:08:59 2020-04-10T07:09:03.943 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror.mirror_status 2020-04-10T07:09:03.944 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror.mirror_status 2020-04-10T07:09:03.944 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:09:03.945 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror.mirror_status 2020-04-10T07:09:03.945 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2020-04-10T07:09:03.988 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: failed to query services: (13) Permission denied 2020-04-10T07:09:04.040 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update state 2020-04-10T07:09:04.041 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2020-04-10T07:09:04.046 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ last_update=2020-04-10 07:08:59 2020-04-10T07:09:04.046 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2020-04-10T07:09:04.056 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ state=up+stopped 2020-04-10T07:09:04.057 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+stopped 2020-04-10T07:09:04.057 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep up+stopped 2020-04-10T07:09:04.057 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:09:04.057 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:09:04.057 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_status_in_pool_dir cluster2 mirror test up+replaying 2020-04-10T07:09:04.057 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:04.058 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:04.058 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:09:04.058 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:09:04.058 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:09:04.058 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:09:04.058 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:09:04.058 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+stopped 2020-04-10T07:09:05.055 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_status_in_pool_dir cluster2 mirror test up+replaying 2020-04-10T07:09:05.055 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:05.055 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:05.055 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:09:05.055 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:09:05.055 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:09:05.055 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:09:05.056 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster2-mirror-test.mirror_status 2020-04-10T07:09:05.056 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2-mirror-test.mirror_status 2020-04-10T07:09:05.057 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:09:05.057 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:09:05.058 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster2 mirror image status mirror/test 2020-04-10T07:09:05.058 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ tee /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:09:05.121 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:test: 2020-04-10T07:09:05.121 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: global_id: a9af7d9e-1404-4730-9e18-c0418a915346 2020-04-10T07:09:05.121 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+replaying 2020-04-10T07:09:05.121 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: replaying, {"replay_state": "idle", "remote_snapshot_timestamp": 1586502532} 2020-04-10T07:09:05.121 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: service: mirror.1 on smithi204 2020-04-10T07:09:05.121 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:08:56 2020-04-10T07:09:05.122 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: peer_sites: 2020-04-10T07:09:05.122 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: name: cluster1-DNE 2020-04-10T07:09:05.122 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+stopped 2020-04-10T07:09:05.122 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: local image is primary 2020-04-10T07:09:05.122 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:09:04 2020-04-10T07:09:05.127 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ state: .*up+replaying /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:09:05.128 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+replaying 2020-04-10T07:09:05.128 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ description: .* /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:09:05.129 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: replaying, {"replay_state": "idle", "remote_snapshot_timestamp": 1586502532} 2020-04-10T07:09:05.129 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:09:05.130 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo+ up+replaying 2020-04-10T07:09:05.130 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:grep ^up+ 2020-04-10T07:09:05.130 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+replaying 2020-04-10T07:09:05.130 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep service: *mirror..* on /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:09:05.131 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: service: mirror.1 on smithi204 2020-04-10T07:09:05.131 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:09:05.134 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update=2020-04-10 07:08:56 2020-04-10T07:09:05.134 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_mirror_pool_status_verbose cluster2 mirror test up+replaying 2020-04-10 07:08:56 2020-04-10T07:09:05.134 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:05.134 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:05.134 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:09:05.135 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:09:05.135 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local prev_last_update=2020-04-10 07:08:56 2020-04-10T07:09:05.135 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster2-mirror.mirror_status 2020-04-10T07:09:05.135 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2-mirror.mirror_status 2020-04-10T07:09:05.135 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:09:05.136 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster2-mirror.mirror_status 2020-04-10T07:09:05.136 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2020-04-10T07:09:05.172 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: failed to query services: (13) Permission denied 2020-04-10T07:09:05.210 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update state 2020-04-10T07:09:05.210 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2020-04-10T07:09:05.213 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ last_update=2020-04-10 07:08:56 2020-04-10T07:09:05.213 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2020-04-10T07:09:05.215 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ state=up+replaying 2020-04-10T07:09:05.215 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+replaying 2020-04-10T07:09:05.216 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep up+replaying 2020-04-10T07:09:05.218 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+replaying 2020-04-10T07:09:05.218 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:09:05.218 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:09:05.218 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ compare_images mirror test 2020-04-10T07:09:05.219 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:05.219 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:09:05.219 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local ret=0 2020-04-10T07:09:05.219 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster2-mirror-test.export 2020-04-10T07:09:05.219 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2-mirror-test.export 2020-04-10T07:09:05.219 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:09:05.219 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local rmt_export=/tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.export 2020-04-10T07:09:05.220 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror-test.export 2020-04-10T07:09:05.220 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror-test.export 2020-04-10T07:09:05.220 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:09:05.220 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local loc_export=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.export 2020-04-10T07:09:05.221 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rm -f /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.export /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.export 2020-04-10T07:09:05.221 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 export mirror/test /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.export 2020-04-10T07:09:05.398 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2020-04-10T07:09:05.402 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 export mirror/test /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.export 2020-04-10T07:09:05.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2020-04-10T07:09:05.593 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cmp /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.export /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.export 2020-04-10T07:09:05.665 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rm -f /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.export /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.export 2020-04-10T07:09:05.705 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:09:05.705 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ demote_image cluster1 mirror test 2020-04-10T07:09:05.705 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:05.705 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:05.705 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:09:05.706 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/test 2020-04-10T07:09:06.151 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Image demoted to non-primary 2020-04-10T07:09:06.163 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_stopped cluster2 mirror test 2020-04-10T07:09:06.163 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:06.163 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:06.163 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:09:06.163 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_state cluster2 mirror test stopped 2020-04-10T07:09:06.164 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:06.164 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:06.164 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:09:06.164 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state=stopped 2020-04-10T07:09:06.164 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local s 2020-04-10T07:09:06.164 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:09:07.164 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_image_replay_state cluster2 mirror test stopped 2020-04-10T07:09:07.165 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:07.165 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:07.165 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:09:07.165 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local test_state=stopped 2020-04-10T07:09:07.165 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_result 2020-04-10T07:09:07.165 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_state=stopped 2020-04-10T07:09:07.165 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemons cluster2 rbd mirror status mirror/test 2020-04-10T07:09:07.166 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ + local cluster_instance=cluster2 2020-04-10T07:09:07.166 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:grep -i state 2020-04-10T07:09:07.166 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:09:07.166 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:07.166 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance=cluster2 2020-04-10T07:09:07.166 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local loop_instance 2020-04-10T07:09:07.166 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0 2020-04-10T07:09:07.171 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ cluster2 != cluster2 ] 2020-04-10T07:09:07.171 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:09:07.171 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster2:0 rbd mirror status mirror/test 2020-04-10T07:09:07.171 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2:0 2020-04-10T07:09:07.171 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:09:07.172 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:09:07.172 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:0 cluster instance 2020-04-10T07:09:07.172 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:0 2020-04-10T07:09:07.172 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:09:07.172 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:09:07.172 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:09:07.172 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:09:07.173 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster2:0 ] 2020-04-10T07:09:07.173 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster2 2020-04-10T07:09:07.173 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:09:07.173 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:09:07.173 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:09:07.173 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster2:0 cluster2 2020-04-10T07:09:07.173 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster2:0 2020-04-10T07:09:07.173 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:07.174 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:09:07.174 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:0 local_cluster instance 2020-04-10T07:09:07.174 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:0 2020-04-10T07:09:07.174 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:09:07.174 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:09:07.174 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:09:07.174 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:09:07.175 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster2:0 ] 2020-04-10T07:09:07.175 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster2 2020-04-10T07:09:07.175 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster2 2020-04-10T07:09:07.175 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:09:07.175 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:09:07.175 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster2 --name client.mirror.0 admin socket 2020-04-10T07:09:07.189 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster2-client.mirror.0.asok 2020-04-10T07:09:07.189 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster2-client.mirror.0.asok 2020-04-10T07:09:07.189 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster2-client.mirror.0.asok 2020-04-10T07:09:07.189 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster2-client.mirror.0.asok rbd mirror status mirror/test 2020-04-10T07:09:07.267 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:no valid command found; 1 closest matches: 2020-04-10T07:09:07.267 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status mirror/test1 2020-04-10T07:09:07.267 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:09:07.270 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster2:1 rbd mirror status mirror/test 2020-04-10T07:09:07.270 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2:1 2020-04-10T07:09:07.270 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:09:07.270 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:09:07.270 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:1 cluster instance 2020-04-10T07:09:07.271 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:1 2020-04-10T07:09:07.271 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:09:07.271 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:09:07.271 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:09:07.271 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:09:07.271 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster2:1 ] 2020-04-10T07:09:07.271 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster2 2020-04-10T07:09:07.272 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:09:07.272 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:09:07.272 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:09:07.272 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster2:1 cluster2 2020-04-10T07:09:07.272 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster2:1 2020-04-10T07:09:07.272 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:07.273 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:09:07.273 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:1 local_cluster instance 2020-04-10T07:09:07.273 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:1 2020-04-10T07:09:07.273 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:09:07.273 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:09:07.273 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:09:07.273 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:09:07.274 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster2:1 ] 2020-04-10T07:09:07.274 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster2 2020-04-10T07:09:07.274 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster2 2020-04-10T07:09:07.274 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:09:07.274 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:09:07.274 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster2 --name client.mirror.1 admin socket 2020-04-10T07:09:07.287 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster2-client.mirror.1.asok 2020-04-10T07:09:07.288 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster2-client.mirror.1.asok 2020-04-10T07:09:07.288 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster2-client.mirror.1.asok 2020-04-10T07:09:07.288 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster2-client.mirror.1.asok rbd mirror status mirror/test 2020-04-10T07:09:07.384 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:09:07.384 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ status_result= "state": "Stopped" 2020-04-10T07:09:07.384 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo "state": "Stopped" 2020-04-10T07:09:07.385 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep -i Replaying 2020-04-10T07:09:07.386 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test stopped = stopped 2020-04-10T07:09:07.386 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:09:07.386 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+unknown 2020-04-10T07:09:07.386 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:07.386 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:07.387 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:09:07.387 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+unknown 2020-04-10T07:09:07.387 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:09:07.387 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:09:07.387 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:09:08.387 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_status_in_pool_dir cluster1 mirror test up+unknown 2020-04-10T07:09:08.388 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:08.388 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:08.388 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:09:08.388 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+unknown 2020-04-10T07:09:08.388 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:09:08.388 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:09:08.388 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror-test.mirror_status 2020-04-10T07:09:08.389 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror-test.mirror_status 2020-04-10T07:09:08.389 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:09:08.390 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:09:08.390 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ + CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test 2020-04-10T07:09:08.390 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:tee /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:09:08.461 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:test: 2020-04-10T07:09:08.461 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: global_id: a9af7d9e-1404-4730-9e18-c0418a915346 2020-04-10T07:09:08.462 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+stopped 2020-04-10T07:09:08.462 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: local image is primary 2020-04-10T07:09:08.462 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: service: mirror.2 on smithi204 2020-04-10T07:09:08.462 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:09:04 2020-04-10T07:09:08.462 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: peer_sites: 2020-04-10T07:09:08.462 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: name: cluster2-DNE 2020-04-10T07:09:08.462 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+error 2020-04-10T07:09:08.462 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: failed to unlink local peer from remote image 2020-04-10T07:09:08.463 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:09:07 2020-04-10T07:09:08.469 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ state: .*up+unknown /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:09:08.471 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 1 2020-04-10T07:09:08.471 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 2 2020-04-10T07:09:10.473 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_status_in_pool_dir cluster1 mirror test up+unknown 2020-04-10T07:09:10.473 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:10.473 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:10.473 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:09:10.473 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+unknown 2020-04-10T07:09:10.474 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:09:10.474 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:09:10.474 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror-test.mirror_status 2020-04-10T07:09:10.474 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror-test.mirror_status 2020-04-10T07:09:10.474 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:09:10.476 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:09:10.476 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test 2020-04-10T07:09:10.476 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ tee /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:09:10.553 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:test: 2020-04-10T07:09:10.553 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: global_id: a9af7d9e-1404-4730-9e18-c0418a915346 2020-04-10T07:09:10.553 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+unknown 2020-04-10T07:09:10.553 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: remote image demoted 2020-04-10T07:09:10.553 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: service: mirror.2 on smithi204 2020-04-10T07:09:10.554 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:09:09 2020-04-10T07:09:10.554 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: peer_sites: 2020-04-10T07:09:10.554 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: name: cluster2-DNE 2020-04-10T07:09:10.554 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+error 2020-04-10T07:09:10.554 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: failed to unlink local peer from remote image 2020-04-10T07:09:10.554 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:09:07 2020-04-10T07:09:10.560 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ state: .*up+unknown /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:09:10.562 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+unknown 2020-04-10T07:09:10.562 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ description: .* /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:09:10.564 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: remote image demoted 2020-04-10T07:09:10.565 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:09:10.565 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+unknown 2020-04-10T07:09:10.565 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^up+ 2020-04-10T07:09:10.566 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+unknown 2020-04-10T07:09:10.566 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep service: *mirror..* on /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:09:10.568 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: service: mirror.2 on smithi204 2020-04-10T07:09:10.568 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:09:10.571 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update=2020-04-10 07:09:09 2020-04-10T07:09:10.571 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+unknown 2020-04-10 07:09:09 2020-04-10T07:09:10.571 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:10.571 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:10.572 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:09:10.572 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+unknown 2020-04-10T07:09:10.572 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local prev_last_update=2020-04-10 07:09:09 2020-04-10T07:09:10.572 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror.mirror_status 2020-04-10T07:09:10.572 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror.mirror_status 2020-04-10T07:09:10.572 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:09:10.573 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror.mirror_status 2020-04-10T07:09:10.573 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2020-04-10T07:09:10.613 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: failed to query services: (13) Permission denied 2020-04-10T07:09:10.667 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update state 2020-04-10T07:09:10.667 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2020-04-10T07:09:10.673 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ last_update=2020-04-10 07:09:09 2020-04-10T07:09:10.673 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2020-04-10T07:09:10.677 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ state=up+unknown 2020-04-10T07:09:10.678 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+unknown 2020-04-10T07:09:10.678 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep up+unknown 2020-04-10T07:09:10.679 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+unknown 2020-04-10T07:09:10.679 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:09:10.679 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:09:10.680 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_status_in_pool_dir cluster2 mirror test up+unknown 2020-04-10T07:09:10.680 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:10.680 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:10.680 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:09:10.680 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+unknown 2020-04-10T07:09:10.680 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:09:10.680 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:09:10.681 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:09:11.682 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_status_in_pool_dir cluster2 mirror test up+unknown 2020-04-10T07:09:11.682 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:11.682 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:11.682 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:09:11.683 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+unknown 2020-04-10T07:09:11.683 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:09:11.683 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:09:11.683 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster2-mirror-test.mirror_status 2020-04-10T07:09:11.683 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2-mirror-test.mirror_status 2020-04-10T07:09:11.683 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:09:11.684 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:09:11.684 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster2 mirror image status mirror/test 2020-04-10T07:09:11.684 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ tee /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:09:11.760 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:test: 2020-04-10T07:09:11.761 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: global_id: a9af7d9e-1404-4730-9e18-c0418a915346 2020-04-10T07:09:11.761 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+unknown 2020-04-10T07:09:11.761 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: remote image demoted 2020-04-10T07:09:11.761 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: service: mirror.1 on smithi204 2020-04-10T07:09:11.761 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:09:11 2020-04-10T07:09:11.761 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: peer_sites: 2020-04-10T07:09:11.762 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: name: cluster1-DNE 2020-04-10T07:09:11.762 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+unknown 2020-04-10T07:09:11.762 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: remote image demoted 2020-04-10T07:09:11.762 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:09:09 2020-04-10T07:09:11.767 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ state: .*up+unknown /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:09:11.768 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+unknown 2020-04-10T07:09:11.768 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ description: .* /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:09:11.769 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: remote image demoted 2020-04-10T07:09:11.770 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:09:11.770 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+unknown 2020-04-10T07:09:11.771 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^up+ 2020-04-10T07:09:11.771 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep service: *mirror..* on /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:09:11.771 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+unknown 2020-04-10T07:09:11.771 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: service: mirror.1 on smithi204 2020-04-10T07:09:11.772 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:09:11.774 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update=2020-04-10 07:09:11 2020-04-10T07:09:11.775 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_mirror_pool_status_verbose cluster2 mirror test up+unknown 2020-04-10 07:09:11 2020-04-10T07:09:11.775 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:11.775 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:11.775 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:09:11.775 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+unknown 2020-04-10T07:09:11.775 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local prev_last_update=2020-04-10 07:09:11 2020-04-10T07:09:11.775 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster2-mirror.mirror_status 2020-04-10T07:09:11.776 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2-mirror.mirror_status 2020-04-10T07:09:11.776 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:09:11.780 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster2-mirror.mirror_status 2020-04-10T07:09:11.781 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2020-04-10T07:09:11.815 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: failed to query services: (13) Permission denied 2020-04-10T07:09:11.857 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update state 2020-04-10T07:09:11.858 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2020-04-10T07:09:11.860 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ last_update=2020-04-10 07:09:11 2020-04-10T07:09:11.860 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2020-04-10T07:09:11.863 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ state=up+unknown 2020-04-10T07:09:11.863 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+unknown 2020-04-10T07:09:11.863 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep up+unknown 2020-04-10T07:09:11.864 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+unknown 2020-04-10T07:09:11.864 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:09:11.864 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:09:11.865 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ promote_image cluster2 mirror test 2020-04-10T07:09:11.865 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:11.865 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:11.865 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:09:11.865 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local force= 2020-04-10T07:09:11.865 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/test 2020-04-10T07:09:12.109 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Image promoted to primary 2020-04-10T07:09:12.119 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_started cluster1 mirror test 2020-04-10T07:09:12.119 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:12.119 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:12.120 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:09:12.120 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2020-04-10T07:09:12.120 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:12.120 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:12.120 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:09:12.120 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state=started 2020-04-10T07:09:12.120 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local s 2020-04-10T07:09:12.121 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:09:13.121 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_image_replay_state cluster1 mirror test started 2020-04-10T07:09:13.121 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:13.121 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:13.121 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:09:13.121 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local test_state=started 2020-04-10T07:09:13.121 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_result 2020-04-10T07:09:13.122 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_state=stopped 2020-04-10T07:09:13.123 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2020-04-10T07:09:13.123 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ + local cluster_instance=cluster1 2020-04-10T07:09:13.124 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:09:13.124 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:grep -i state 2020-04-10T07:09:13.124 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:13.124 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance=cluster1 2020-04-10T07:09:13.124 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local loop_instance 2020-04-10T07:09:13.124 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0 2020-04-10T07:09:13.125 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ cluster1 != cluster1 ] 2020-04-10T07:09:13.125 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:09:13.127 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2020-04-10T07:09:13.127 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:09:13.128 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:09:13.128 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:09:13.128 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:09:13.128 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:09:13.128 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:09:13.128 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:09:13.129 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:09:13.129 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:09:13.129 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:09:13.129 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:09:13.129 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:09:13.129 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:09:13.130 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:09:13.130 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster1 2020-04-10T07:09:13.130 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:09:13.130 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:13.130 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:09:13.130 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:09:13.131 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:09:13.131 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:09:13.131 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:09:13.131 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:09:13.131 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:09:13.131 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:09:13.131 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:09:13.132 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:09:13.132 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:09:13.132 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:09:13.132 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:09:13.146 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:09:13.146 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:09:13.146 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:09:13.146 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2020-04-10T07:09:13.241 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:no valid command found; 1 closest matches: 2020-04-10T07:09:13.242 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status mirror/test1 2020-04-10T07:09:13.242 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:09:13.244 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2020-04-10T07:09:13.245 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:1 2020-04-10T07:09:13.245 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:09:13.245 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:09:13.245 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 cluster instance 2020-04-10T07:09:13.245 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:09:13.245 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:09:13.246 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:09:13.246 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:09:13.246 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:09:13.246 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:09:13.246 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:09:13.246 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:09:13.247 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:09:13.247 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:09:13.247 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:1 cluster1 2020-04-10T07:09:13.247 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:1 2020-04-10T07:09:13.247 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:13.247 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:09:13.247 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2020-04-10T07:09:13.248 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:09:13.248 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:09:13.248 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:09:13.248 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:09:13.248 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:09:13.248 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:09:13.249 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:09:13.249 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:09:13.249 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:09:13.249 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:09:13.249 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2020-04-10T07:09:13.265 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:09:13.265 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:09:13.266 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:09:13.266 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2020-04-10T07:09:13.361 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:Invalid command: unused arguments: ['mirror/test'] 2020-04-10T07:09:13.361 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status : get status for rbd mirror 2020-04-10T07:09:13.361 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:09:13.363 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:2 rbd mirror status mirror/test 2020-04-10T07:09:13.363 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:2 2020-04-10T07:09:13.363 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:09:13.364 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:09:13.364 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 cluster instance 2020-04-10T07:09:13.364 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:09:13.364 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:09:13.364 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:09:13.364 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:09:13.364 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:09:13.367 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:09:13.367 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:09:13.368 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:09:13.368 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:09:13.368 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:09:13.368 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:2 cluster1 2020-04-10T07:09:13.368 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:2 2020-04-10T07:09:13.368 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:13.368 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:09:13.369 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2020-04-10T07:09:13.369 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:09:13.369 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:09:13.369 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:09:13.369 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:09:13.369 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:09:13.369 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:09:13.370 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:09:13.370 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:09:13.370 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:09:13.370 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:09:13.370 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2020-04-10T07:09:13.385 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:09:13.385 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:09:13.385 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:09:13.386 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2020-04-10T07:09:13.482 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:09:13.483 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ status_result= "state": "Stopped" 2020-04-10T07:09:13.483 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo "state": "Stopped" 2020-04-10T07:09:13.483 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep -i Replaying 2020-04-10T07:09:13.484 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test started = stopped 2020-04-10T07:09:13.484 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 2 2020-04-10T07:09:15.486 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_image_replay_state cluster1 mirror test started 2020-04-10T07:09:15.486 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:15.486 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:15.486 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:09:15.486 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local test_state=started 2020-04-10T07:09:15.486 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_result 2020-04-10T07:09:15.487 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_state=stopped 2020-04-10T07:09:15.488 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2020-04-10T07:09:15.488 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_instance=cluster1 2020-04-10T07:09:15.488 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:09:15.488 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local+ cluster=cluster1 2020-04-10T07:09:15.488 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance=cluster1 2020-04-10T07:09:15.489 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:grep -i+ statelocal 2020-04-10T07:09:15.489 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: loop_instance 2020-04-10T07:09:15.489 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0 2020-04-10T07:09:15.489 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ cluster1 != cluster1 ] 2020-04-10T07:09:15.489 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:09:15.491 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2020-04-10T07:09:15.491 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:09:15.491 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:09:15.491 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:09:15.491 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:09:15.491 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:09:15.491 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:09:15.492 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:09:15.492 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:09:15.492 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:09:15.492 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:09:15.493 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:09:15.493 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:09:15.493 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:09:15.494 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:09:15.494 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster1 2020-04-10T07:09:15.494 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:09:15.494 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:15.494 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:09:15.494 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:09:15.494 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:09:15.495 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:09:15.496 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:09:15.496 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:09:15.496 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:09:15.496 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:09:15.496 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:09:15.496 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:09:15.496 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:09:15.497 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:09:15.497 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:09:15.514 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:09:15.514 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:09:15.514 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:09:15.514 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2020-04-10T07:09:15.611 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:no valid command found; 1 closest matches: 2020-04-10T07:09:15.611 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status mirror/test1 2020-04-10T07:09:15.611 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:09:15.618 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2020-04-10T07:09:15.619 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:1 2020-04-10T07:09:15.619 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:09:15.619 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:09:15.619 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 cluster instance 2020-04-10T07:09:15.619 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:09:15.619 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:09:15.619 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:09:15.620 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:09:15.620 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:09:15.620 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:09:15.620 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:09:15.620 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:09:15.620 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:09:15.620 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:09:15.621 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:1 cluster1 2020-04-10T07:09:15.621 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:1 2020-04-10T07:09:15.621 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:15.621 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:09:15.621 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2020-04-10T07:09:15.621 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:09:15.621 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:09:15.621 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:09:15.622 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:09:15.622 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:09:15.622 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:09:15.622 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:09:15.622 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:09:15.622 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:09:15.622 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:09:15.623 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2020-04-10T07:09:15.634 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:09:15.634 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:09:15.634 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:09:15.634 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2020-04-10T07:09:15.726 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:Invalid command: unused arguments: ['mirror/test'] 2020-04-10T07:09:15.726 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status : get status for rbd mirror 2020-04-10T07:09:15.726 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:09:15.729 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:2 rbd mirror status mirror/test 2020-04-10T07:09:15.729 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:2 2020-04-10T07:09:15.729 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:09:15.729 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:09:15.729 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 cluster instance 2020-04-10T07:09:15.729 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:09:15.730 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:09:15.730 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:09:15.730 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:09:15.730 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:09:15.730 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:09:15.731 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:09:15.731 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:09:15.731 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:09:15.731 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:09:15.731 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:2 cluster1 2020-04-10T07:09:15.731 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:2 2020-04-10T07:09:15.731 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:15.732 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:09:15.732 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2020-04-10T07:09:15.732 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:09:15.732 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:09:15.732 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:09:15.733 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:09:15.733 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:09:15.733 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:09:15.733 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:09:15.733 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:09:15.733 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:09:15.733 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:09:15.734 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2020-04-10T07:09:15.749 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:09:15.749 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:09:15.749 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:09:15.749 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2020-04-10T07:09:15.844 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:09:15.845 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ status_result= "state": "Replaying" 2020-04-10T07:09:15.845 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo "state": "Replaying" 2020-04-10T07:09:15.845 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep -i Replaying 2020-04-10T07:09:15.847 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "state": "Replaying" 2020-04-10T07:09:15.847 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ current_state=started 2020-04-10T07:09:15.848 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test started = started 2020-04-10T07:09:15.848 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:09:15.848 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ write_image cluster2 mirror test 100 2020-04-10T07:09:15.848 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:15.848 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:15.848 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:09:15.849 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local count=100 2020-04-10T07:09:15.849 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local size= 2020-04-10T07:09:15.849 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 2020-04-10T07:09:15.849 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ size=4096 2020-04-10T07:09:15.849 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 bench mirror/test --io-type write --io-size 4096 --io-threads 1 --io-total 409600 --io-pattern rand 2020-04-10T07:09:15.928 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:bench type write io_size 4096 io_threads 1 bytes 409600 pattern random 2020-04-10T07:09:16.000 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: SEC OPS OPS/SEC BYTES/SEC 2020-04-10T07:09:16.000 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:elapsed: 0 ops: 100 ops/sec: 1388.89 bytes/sec: 5.4 MiB/s 2020-04-10T07:09:16.013 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror test 2020-04-10T07:09:16.013 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1 2020-04-10T07:09:16.014 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:16.014 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:16.014 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:09:16.014 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ snapshot = journal ] 2020-04-10T07:09:16.014 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ snapshot = snapshot ] 2020-04-10T07:09:16.014 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror test 2020-04-10T07:09:16.014 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1 2020-04-10T07:09:16.015 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:16.015 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:16.015 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:09:16.015 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster2-mirror-test.status 2020-04-10T07:09:16.015 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2-mirror-test.status 2020-04-10T07:09:16.015 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:09:16.015 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.status 2020-04-10T07:09:16.016 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror-test.status 2020-04-10T07:09:16.016 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror-test.status 2020-04-10T07:09:16.016 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:09:16.017 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.status 2020-04-10T07:09:16.017 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mirror_image_snapshot cluster2 mirror test 2020-04-10T07:09:16.017 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:16.017 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:16.017 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:09:16.017 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test 2020-04-10T07:09:17.128 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Snapshot ID: 34 2020-04-10T07:09:17.138 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_newest_mirror_snapshot cluster2 mirror test /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.status 2020-04-10T07:09:17.139 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:17.139 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:17.139 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:09:17.139 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local log=/tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.status 2020-04-10T07:09:17.139 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 snap list --all mirror/test --format xml 2020-04-10T07:09:17.139 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2020-04-10T07:09:17.193 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //snapshot/id 2020-04-10T07:09:17.197 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snapshot_id=34 2020-04-10T07:09:17.198 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ true 2020-04-10T07:09:17.198 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0.2 2020-04-10T07:09:17.400 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.status 2020-04-10T07:09:17.400 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:17.400 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:17.400 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:09:17.400 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.status 2020-04-10T07:09:17.400 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2020-04-10T07:09:17.401 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2020-04-10T07:09:17.476 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2020-04-10T07:09:17.480 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local primary_snapshot_id=33 2020-04-10T07:09:17.481 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test 34 = 33 2020-04-10T07:09:17.481 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0.4 2020-04-10T07:09:17.882 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.status 2020-04-10T07:09:17.882 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:17.882 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:17.883 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:09:17.883 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.status 2020-04-10T07:09:17.883 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2020-04-10T07:09:17.883 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2020-04-10T07:09:17.949 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2020-04-10T07:09:17.955 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local primary_snapshot_id=34 2020-04-10T07:09:17.955 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test 34 = 34 2020-04-10T07:09:17.955 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:09:17.955 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2020-04-10T07:09:17.955 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:17.955 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:17.956 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:09:17.956 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:09:17.956 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:09:17.956 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:09:17.956 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:09:18.956 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying 2020-04-10T07:09:18.956 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:18.956 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:18.957 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:09:18.957 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:09:18.957 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:09:18.957 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:09:18.957 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror-test.mirror_status 2020-04-10T07:09:18.958 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror-test.mirror_status 2020-04-10T07:09:18.958 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:09:18.959 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:09:18.959 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test 2020-04-10T07:09:18.960 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ tee /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:09:19.037 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:test: 2020-04-10T07:09:19.037 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: global_id: a9af7d9e-1404-4730-9e18-c0418a915346 2020-04-10T07:09:19.037 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+replaying 2020-04-10T07:09:19.037 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: replaying, {"replay_state": "idle", "remote_snapshot_timestamp": 1586502552} 2020-04-10T07:09:19.037 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: service: mirror.2 on smithi204 2020-04-10T07:09:19.038 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:09:14 2020-04-10T07:09:19.038 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: peer_sites: 2020-04-10T07:09:19.038 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: name: cluster2-DNE 2020-04-10T07:09:19.038 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+stopped 2020-04-10T07:09:19.038 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: local image is primary 2020-04-10T07:09:19.038 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:09:16 2020-04-10T07:09:19.053 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ state: .*up+replaying /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:09:19.054 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ description: .* /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:09:19.054 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:09:19.054 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+replaying 2020-04-10T07:09:19.054 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^up+ 2020-04-10T07:09:19.054 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+replaying 2020-04-10T07:09:19.054 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: replaying, {"replay_state": "idle", "remote_snapshot_timestamp": 1586502552} 2020-04-10T07:09:19.054 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+replaying 2020-04-10T07:09:19.056 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep service: *mirror..* on /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:09:19.056 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.mirror_status 2020-04-10T07:09:19.056 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update=2020-04-10 07:09:14 2020-04-10T07:09:19.056 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying 2020-04-10 07:09:14 2020-04-10T07:09:19.057 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:19.057 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:19.057 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:09:19.057 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:09:19.057 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local prev_last_update=2020-04-10 07:09:14 2020-04-10T07:09:19.057 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror.mirror_status 2020-04-10T07:09:19.057 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror.mirror_status 2020-04-10T07:09:19.058 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:09:19.058 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: service: mirror.2 on smithi204 2020-04-10T07:09:19.058 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror.mirror_status 2020-04-10T07:09:19.058 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2020-04-10T07:09:19.100 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: failed to query services: (13) Permission denied 2020-04-10T07:09:19.151 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update state 2020-04-10T07:09:19.152 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2020-04-10T07:09:19.157 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ last_update=2020-04-10 07:09:14 2020-04-10T07:09:19.158 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2020-04-10T07:09:19.161 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ state=up+replaying 2020-04-10T07:09:19.162 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+replaying 2020-04-10T07:09:19.162 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep up+replaying 2020-04-10T07:09:19.163 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+replaying 2020-04-10T07:09:19.164 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:09:19.164 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:09:19.164 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_status_in_pool_dir cluster2 mirror test up+stopped 2020-04-10T07:09:19.164 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:19.164 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:19.164 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:09:19.165 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+stopped 2020-04-10T07:09:19.165 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:09:19.165 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:09:19.165 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:09:20.166 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_status_in_pool_dir cluster2 mirror test up+stopped 2020-04-10T07:09:20.166 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:20.166 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:20.166 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:09:20.166 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+stopped 2020-04-10T07:09:20.166 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:09:20.167 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:09:20.167 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster2-mirror-test.mirror_status 2020-04-10T07:09:20.167 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2-mirror-test.mirror_status 2020-04-10T07:09:20.167 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:09:20.167 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:09:20.168 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster2 mirror image status mirror/test 2020-04-10T07:09:20.168 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ tee /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:09:20.239 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:test: 2020-04-10T07:09:20.239 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: global_id: a9af7d9e-1404-4730-9e18-c0418a915346 2020-04-10T07:09:20.240 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+stopped 2020-04-10T07:09:20.240 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: local image is primary 2020-04-10T07:09:20.240 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: service: mirror.1 on smithi204 2020-04-10T07:09:20.240 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:09:16 2020-04-10T07:09:20.240 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: peer_sites: 2020-04-10T07:09:20.240 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: name: cluster1-DNE 2020-04-10T07:09:20.241 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+replaying 2020-04-10T07:09:20.241 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: replaying, {"replay_state": "idle", "remote_snapshot_timestamp": 1586502552} 2020-04-10T07:09:20.241 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:09:14 2020-04-10T07:09:20.241 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: snapshots: 2020-04-10T07:09:20.241 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: 17 .mirror.primary.a9af7d9e-1404-4730-9e18-c0418a915346.ae5a942a-32bb-40d8-b4b6-4b6da2b8780c (peer_uuids:[90b6cfa2-5481-4edf-b462-5494aab62f6f]) 2020-04-10T07:09:20.241 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: 21 .mirror.primary.a9af7d9e-1404-4730-9e18-c0418a915346.720d033f-30db-4fdf-9736-fd21d1dfaf57 (peer_uuids:[90b6cfa2-5481-4edf-b462-5494aab62f6f]) 2020-04-10T07:09:20.241 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: 23 .mirror.primary.a9af7d9e-1404-4730-9e18-c0418a915346.89218af1-114d-4250-8a2f-0710ac21a25a (demoted peer_uuids:[90b6cfa2-5481-4edf-b462-5494aab62f6f]) 2020-04-10T07:09:20.242 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: 27 .mirror.primary.a9af7d9e-1404-4730-9e18-c0418a915346.56964f43-33bc-42bb-89f8-032b8753f629 (peer_uuids:[90b6cfa2-5481-4edf-b462-5494aab62f6f]) 2020-04-10T07:09:20.242 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: 29 .mirror.primary.a9af7d9e-1404-4730-9e18-c0418a915346.f378808d-4c4b-49ac-9f7e-0db6752f7300 (demoted peer_uuids:[90b6cfa2-5481-4edf-b462-5494aab62f6f]) 2020-04-10T07:09:20.242 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: 34 .mirror.primary.a9af7d9e-1404-4730-9e18-c0418a915346.24af8544-2fd5-448a-a928-daaa20954150 (peer_uuids:[90b6cfa2-5481-4edf-b462-5494aab62f6f]) 2020-04-10T07:09:20.246 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ state: .*up+stopped /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:09:20.248 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+stopped 2020-04-10T07:09:20.248 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ description: .* /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:09:20.250 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: local image is primary 2020-04-10T07:09:20.250 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:09:20.250 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ + grep ^up+ 2020-04-10T07:09:20.250 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:echo up+stopped 2020-04-10T07:09:20.251 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+stopped 2020-04-10T07:09:20.251 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep service: *mirror..* on /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:09:20.252 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: service: mirror.1 on smithi204 2020-04-10T07:09:20.253 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.mirror_status 2020-04-10T07:09:20.255 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update=2020-04-10 07:09:16 2020-04-10T07:09:20.255 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_mirror_pool_status_verbose cluster2 mirror test up+stopped 2020-04-10 07:09:16 2020-04-10T07:09:20.255 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:20.255 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:20.255 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:09:20.256 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+stopped 2020-04-10T07:09:20.256 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local prev_last_update=2020-04-10 07:09:16 2020-04-10T07:09:20.256 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster2-mirror.mirror_status 2020-04-10T07:09:20.256 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2-mirror.mirror_status 2020-04-10T07:09:20.256 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:09:20.257 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster2-mirror.mirror_status 2020-04-10T07:09:20.257 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2020-04-10T07:09:20.294 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: failed to query services: (13) Permission denied 2020-04-10T07:09:20.337 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update state 2020-04-10T07:09:20.337 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2020-04-10T07:09:20.340 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ last_update=2020-04-10 07:09:16 2020-04-10T07:09:20.340 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2020-04-10T07:09:20.342 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ state=up+stopped 2020-04-10T07:09:20.342 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+stopped 2020-04-10T07:09:20.343 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep up+stopped 2020-04-10T07:09:20.344 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+stopped 2020-04-10T07:09:20.344 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:09:20.344 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:09:20.345 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ compare_images mirror test 2020-04-10T07:09:20.345 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:20.345 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test 2020-04-10T07:09:20.345 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local ret=0 2020-04-10T07:09:20.345 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster2-mirror-test.export 2020-04-10T07:09:20.345 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2-mirror-test.export 2020-04-10T07:09:20.345 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:09:20.347 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local rmt_export=/tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.export 2020-04-10T07:09:20.347 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror-test.export 2020-04-10T07:09:20.347 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror-test.export 2020-04-10T07:09:20.347 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:09:20.347 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local loc_export=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.export 2020-04-10T07:09:20.347 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rm -f /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.export /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.export 2020-04-10T07:09:20.347 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 export mirror/test /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.export 2020-04-10T07:09:20.519 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2020-04-10T07:09:20.522 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 export mirror/test /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.export 2020-04-10T07:09:20.716 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2020-04-10T07:09:20.720 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cmp /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.export /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.export 2020-04-10T07:09:20.793 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rm -f /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test.export /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test.export 2020-04-10T07:09:20.831 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:09:20.831 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ force_promote_image=test_force_promote 2020-04-10T07:09:20.831 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ create_image_and_enable_mirror cluster2 mirror test_force_promote 2020-04-10T07:09:20.831 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:20.831 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:09:20.831 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:20.831 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:09:20.832 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_force_promote 2020-04-10T07:09:20.832 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:09:20.832 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local mode=snapshot 2020-04-10T07:09:20.832 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:09:20.832 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ create_image cluster2 mirror test_force_promote 2020-04-10T07:09:20.832 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:20.832 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:09:20.833 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:20.833 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:09:20.833 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_force_promote 2020-04-10T07:09:20.833 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:09:20.833 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local size=128 2020-04-10T07:09:20.833 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:09:20.833 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 create --size 128 --image-feature layering,exclusive-lock,journaling mirror/test_force_promote 2020-04-10T07:09:20.888 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ enable_mirror cluster2 mirror test_force_promote snapshot 2020-04-10T07:09:20.888 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:20.889 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:20.889 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_force_promote 2020-04-10T07:09:20.889 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local mode=snapshot 2020-04-10T07:09:20.889 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/test_force_promote snapshot 2020-04-10T07:09:21.139 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Mirroring enabled 2020-04-10T07:09:21.148 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ write_image cluster2 mirror test_force_promote 100 2020-04-10T07:09:21.148 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:21.149 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:21.149 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_force_promote 2020-04-10T07:09:21.149 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local count=100 2020-04-10T07:09:21.149 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local size= 2020-04-10T07:09:21.149 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 2020-04-10T07:09:21.149 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ size=4096 2020-04-10T07:09:21.149 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 bench mirror/test_force_promote --io-type write --io-size 4096 --io-threads 1 --io-total 409600 --io-pattern rand 2020-04-10T07:09:21.217 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:bench type write io_size 4096 io_threads 1 bytes 409600 pattern random 2020-04-10T07:09:21.280 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: SEC OPS OPS/SEC BYTES/SEC 2020-04-10T07:09:21.280 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:elapsed: 0 ops: 100 ops/sec: 1562.5 bytes/sec: 6.1 MiB/s 2020-04-10T07:09:21.292 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_stopped cluster2 mirror test_force_promote 2020-04-10T07:09:21.292 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:21.292 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:21.292 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_force_promote 2020-04-10T07:09:21.292 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_state cluster2 mirror test_force_promote stopped 2020-04-10T07:09:21.292 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:21.293 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:21.293 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_force_promote 2020-04-10T07:09:21.293 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state=stopped 2020-04-10T07:09:21.293 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local s 2020-04-10T07:09:21.293 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:09:22.292 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_image_replay_state cluster2 mirror test_force_promote stopped 2020-04-10T07:09:22.293 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:22.293 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:22.293 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_force_promote 2020-04-10T07:09:22.293 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local test_state=stopped 2020-04-10T07:09:22.293 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_result 2020-04-10T07:09:22.293 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_state=stopped 2020-04-10T07:09:22.295 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemons cluster2 rbd mirror status mirror/test_force_promote 2020-04-10T07:09:22.295 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local+ cluster_instance=cluster2 2020-04-10T07:09:22.295 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:09:22.295 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:grep -i+ statelocal 2020-04-10T07:09:22.295 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: cluster=cluster2 2020-04-10T07:09:22.295 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance=cluster2 2020-04-10T07:09:22.295 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local loop_instance 2020-04-10T07:09:22.296 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0 2020-04-10T07:09:22.296 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ cluster2 != cluster2 ] 2020-04-10T07:09:22.296 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:09:22.298 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster2:0 rbd mirror status mirror/test_force_promote 2020-04-10T07:09:22.298 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2:0 2020-04-10T07:09:22.298 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:09:22.298 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:09:22.298 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:0 cluster instance 2020-04-10T07:09:22.298 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:0 2020-04-10T07:09:22.298 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:09:22.299 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:09:22.299 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:09:22.299 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:09:22.299 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster2:0 ] 2020-04-10T07:09:22.299 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster2 2020-04-10T07:09:22.299 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:09:22.299 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:09:22.300 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:09:22.300 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster2:0 cluster2 2020-04-10T07:09:22.300 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster2:0 2020-04-10T07:09:22.300 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:22.300 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:09:22.300 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:0 local_cluster instance 2020-04-10T07:09:22.300 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:0 2020-04-10T07:09:22.301 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:09:22.301 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:09:22.301 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:09:22.302 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:09:22.302 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster2:0 ] 2020-04-10T07:09:22.302 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster2 2020-04-10T07:09:22.302 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster2 2020-04-10T07:09:22.302 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:09:22.302 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:09:22.303 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster2 --name client.mirror.0 admin socket 2020-04-10T07:09:22.327 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster2-client.mirror.0.asok 2020-04-10T07:09:22.327 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster2-client.mirror.0.asok 2020-04-10T07:09:22.328 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster2-client.mirror.0.asok 2020-04-10T07:09:22.328 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster2-client.mirror.0.asok rbd mirror status mirror/test_force_promote 2020-04-10T07:09:22.417 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:Invalid command: unused arguments: ['mirror/test_force_promote'] 2020-04-10T07:09:22.417 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status : get status for rbd mirror 2020-04-10T07:09:22.417 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:09:22.417 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster2:1 rbd mirror status mirror/test_force_promote 2020-04-10T07:09:22.418 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2:1 2020-04-10T07:09:22.418 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:09:22.418 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:09:22.418 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:1 cluster instance 2020-04-10T07:09:22.418 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:1 2020-04-10T07:09:22.418 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:09:22.418 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:09:22.419 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:09:22.419 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:09:22.419 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster2:1 ] 2020-04-10T07:09:22.419 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster2 2020-04-10T07:09:22.419 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:09:22.419 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:09:22.419 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:09:22.420 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster2:1 cluster2 2020-04-10T07:09:22.420 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster2:1 2020-04-10T07:09:22.420 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:22.420 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:09:22.420 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:1 local_cluster instance 2020-04-10T07:09:22.420 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:1 2020-04-10T07:09:22.420 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:09:22.421 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:09:22.421 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:09:22.421 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:09:22.421 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster2:1 ] 2020-04-10T07:09:22.421 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster2 2020-04-10T07:09:22.421 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster2 2020-04-10T07:09:22.421 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:09:22.421 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:09:22.422 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster2 --name client.mirror.1 admin socket 2020-04-10T07:09:22.438 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster2-client.mirror.1.asok 2020-04-10T07:09:22.439 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster2-client.mirror.1.asok 2020-04-10T07:09:22.439 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster2-client.mirror.1.asok 2020-04-10T07:09:22.439 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster2-client.mirror.1.asok rbd mirror status mirror/test_force_promote 2020-04-10T07:09:22.528 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:Invalid command: unused arguments: ['mirror/test_force_promote'] 2020-04-10T07:09:22.529 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status : get status for rbd mirror 2020-04-10T07:09:22.529 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:09:22.530 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster2:2 rbd mirror status mirror/test_force_promote 2020-04-10T07:09:22.531 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2:2 2020-04-10T07:09:22.531 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:09:22.531 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:09:22.531 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:2 cluster instance 2020-04-10T07:09:22.531 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:2 2020-04-10T07:09:22.531 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:09:22.532 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:09:22.532 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:09:22.532 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:09:22.532 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster2:2 ] 2020-04-10T07:09:22.532 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster2 2020-04-10T07:09:22.532 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:09:22.532 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:09:22.533 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:09:22.533 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster2:2 cluster2 2020-04-10T07:09:22.533 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster2:2 2020-04-10T07:09:22.533 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:22.533 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:09:22.533 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:2 local_cluster instance 2020-04-10T07:09:22.533 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:2 2020-04-10T07:09:22.533 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:09:22.534 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:09:22.534 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:09:22.534 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:09:22.534 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster2:2 ] 2020-04-10T07:09:22.534 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster2 2020-04-10T07:09:22.534 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster2 2020-04-10T07:09:22.534 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:09:22.535 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:09:22.535 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster2 --name client.mirror.2 admin socket 2020-04-10T07:09:22.552 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster2-client.mirror.2.asok 2020-04-10T07:09:22.552 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster2-client.mirror.2.asok 2020-04-10T07:09:22.552 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster2-client.mirror.2.asok 2020-04-10T07:09:22.552 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster2-client.mirror.2.asok rbd mirror status mirror/test_force_promote 2020-04-10T07:09:22.638 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:Invalid command: unused arguments: ['mirror/test_force_promote'] 2020-04-10T07:09:22.639 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status : get status for rbd mirror 2020-04-10T07:09:22.639 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:09:22.643 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster2:3 rbd mirror status mirror/test_force_promote 2020-04-10T07:09:22.643 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2:3 2020-04-10T07:09:22.643 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:09:22.644 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:09:22.644 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:3 cluster instance 2020-04-10T07:09:22.644 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:3 2020-04-10T07:09:22.644 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:09:22.644 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:09:22.644 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:09:22.644 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:09:22.645 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 3 = cluster2:3 ] 2020-04-10T07:09:22.645 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster2 2020-04-10T07:09:22.645 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:09:22.645 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=3 2020-04-10T07:09:22.645 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:09:22.645 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster2:3 cluster2 2020-04-10T07:09:22.645 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster2:3 2020-04-10T07:09:22.646 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:22.646 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:09:22.646 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:3 local_cluster instance 2020-04-10T07:09:22.646 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:3 2020-04-10T07:09:22.646 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:09:22.646 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:09:22.646 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:09:22.647 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:09:22.647 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 3 = cluster2:3 ] 2020-04-10T07:09:22.647 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster2 2020-04-10T07:09:22.647 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster2 2020-04-10T07:09:22.647 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=3 2020-04-10T07:09:22.647 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:09:22.647 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster2 --name client.mirror.3 admin socket 2020-04-10T07:09:22.665 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster2-client.mirror.3.asok 2020-04-10T07:09:22.665 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster2-client.mirror.3.asok 2020-04-10T07:09:22.665 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster2-client.mirror.3.asok 2020-04-10T07:09:22.665 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster2-client.mirror.3.asok rbd mirror status mirror/test_force_promote 2020-04-10T07:09:22.746 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:Invalid command: unused arguments: ['mirror/test_force_promote'] 2020-04-10T07:09:22.746 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status : get status for rbd mirror 2020-04-10T07:09:22.746 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:09:22.747 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:09:23.749 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ cluster2 != cluster2 ] 2020-04-10T07:09:23.750 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:09:23.752 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster2:0 rbd mirror status mirror/test_force_promote 2020-04-10T07:09:23.752 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2:0 2020-04-10T07:09:23.752 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:09:23.752 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:09:23.752 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:0 cluster instance 2020-04-10T07:09:23.753 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:0 2020-04-10T07:09:23.753 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:09:23.753 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:09:23.753 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:09:23.753 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:09:23.753 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster2:0 ] 2020-04-10T07:09:23.753 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster2 2020-04-10T07:09:23.754 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:09:23.754 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:09:23.754 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:09:23.754 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster2:0 cluster2 2020-04-10T07:09:23.754 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster2:0 2020-04-10T07:09:23.754 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:23.754 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:09:23.755 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:0 local_cluster instance 2020-04-10T07:09:23.755 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:0 2020-04-10T07:09:23.755 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:09:23.755 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:09:23.755 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:09:23.755 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:09:23.756 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster2:0 ] 2020-04-10T07:09:23.756 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster2 2020-04-10T07:09:23.756 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster2 2020-04-10T07:09:23.756 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:09:23.756 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:09:23.756 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster2 --name client.mirror.0 admin socket 2020-04-10T07:09:23.774 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster2-client.mirror.0.asok 2020-04-10T07:09:23.774 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster2-client.mirror.0.asok 2020-04-10T07:09:23.775 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster2-client.mirror.0.asok 2020-04-10T07:09:23.775 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster2-client.mirror.0.asok rbd mirror status mirror/test_force_promote 2020-04-10T07:09:23.863 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:Invalid command: unused arguments: ['mirror/test_force_promote'] 2020-04-10T07:09:23.863 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status : get status for rbd mirror 2020-04-10T07:09:23.864 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:09:23.864 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster2:1 rbd mirror status mirror/test_force_promote 2020-04-10T07:09:23.864 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2:1 2020-04-10T07:09:23.864 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:09:23.864 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:09:23.864 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:1 cluster instance 2020-04-10T07:09:23.865 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:1 2020-04-10T07:09:23.865 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:09:23.865 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:09:23.865 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:09:23.865 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:09:23.865 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster2:1 ] 2020-04-10T07:09:23.865 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster2 2020-04-10T07:09:23.866 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:09:23.866 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:09:23.866 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:09:23.866 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster2:1 cluster2 2020-04-10T07:09:23.866 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster2:1 2020-04-10T07:09:23.866 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:23.866 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:09:23.867 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:1 local_cluster instance 2020-04-10T07:09:23.867 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:1 2020-04-10T07:09:23.867 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:09:23.867 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:09:23.867 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:09:23.867 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:09:23.867 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster2:1 ] 2020-04-10T07:09:23.867 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster2 2020-04-10T07:09:23.868 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster2 2020-04-10T07:09:23.868 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:09:23.868 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:09:23.868 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster2 --name client.mirror.1 admin socket 2020-04-10T07:09:23.885 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster2-client.mirror.1.asok 2020-04-10T07:09:23.885 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster2-client.mirror.1.asok 2020-04-10T07:09:23.885 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster2-client.mirror.1.asok 2020-04-10T07:09:23.886 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster2-client.mirror.1.asok rbd mirror status mirror/test_force_promote 2020-04-10T07:09:23.977 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:Invalid command: unused arguments: ['mirror/test_force_promote'] 2020-04-10T07:09:23.977 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status : get status for rbd mirror 2020-04-10T07:09:23.977 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:09:23.979 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster2:2 rbd mirror status mirror/test_force_promote 2020-04-10T07:09:23.980 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2:2 2020-04-10T07:09:23.980 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:09:23.980 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:09:23.980 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:2 cluster instance 2020-04-10T07:09:23.980 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:2 2020-04-10T07:09:23.980 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:09:23.980 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:09:23.981 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:09:23.981 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:09:23.981 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster2:2 ] 2020-04-10T07:09:23.981 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster2 2020-04-10T07:09:23.981 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:09:23.981 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:09:23.981 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:09:23.982 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster2:2 cluster2 2020-04-10T07:09:23.982 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster2:2 2020-04-10T07:09:23.982 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:23.982 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:09:23.982 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:2 local_cluster instance 2020-04-10T07:09:23.982 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:2 2020-04-10T07:09:23.982 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:09:23.983 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:09:23.983 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:09:23.983 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:09:23.983 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster2:2 ] 2020-04-10T07:09:23.983 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster2 2020-04-10T07:09:23.983 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster2 2020-04-10T07:09:23.983 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:09:23.984 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:09:23.984 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster2 --name client.mirror.2 admin socket 2020-04-10T07:09:24.000 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster2-client.mirror.2.asok 2020-04-10T07:09:24.001 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster2-client.mirror.2.asok 2020-04-10T07:09:24.001 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster2-client.mirror.2.asok 2020-04-10T07:09:24.001 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster2-client.mirror.2.asok rbd mirror status mirror/test_force_promote 2020-04-10T07:09:24.101 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:09:24.102 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ status_result= "state": "Stopped" 2020-04-10T07:09:24.102 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ + echo "state": "Stopped" 2020-04-10T07:09:24.102 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:grep -i Replaying 2020-04-10T07:09:24.103 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test stopped = stopped 2020-04-10T07:09:24.103 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:09:24.103 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_started cluster1 mirror test_force_promote 2020-04-10T07:09:24.103 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:24.104 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:24.104 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_force_promote 2020-04-10T07:09:24.104 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_state cluster1 mirror test_force_promote started 2020-04-10T07:09:24.104 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:24.104 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:24.104 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_force_promote 2020-04-10T07:09:24.105 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state=started 2020-04-10T07:09:24.105 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local s 2020-04-10T07:09:24.105 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:09:25.106 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_image_replay_state cluster1 mirror test_force_promote started 2020-04-10T07:09:25.106 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:25.106 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:25.106 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_force_promote 2020-04-10T07:09:25.106 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local test_state=started 2020-04-10T07:09:25.106 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_result 2020-04-10T07:09:25.107 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_state=stopped 2020-04-10T07:09:25.107 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test_force_promote 2020-04-10T07:09:25.107 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ + local cluster_instance=cluster1 2020-04-10T07:09:25.107 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep -i stateshift 2020-04-10T07:09:25.107 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: 2020-04-10T07:09:25.107 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:25.107 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance=cluster1 2020-04-10T07:09:25.108 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local loop_instance 2020-04-10T07:09:25.108 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0 2020-04-10T07:09:25.109 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ cluster1 != cluster1 ] 2020-04-10T07:09:25.109 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:09:25.110 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test_force_promote 2020-04-10T07:09:25.110 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:09:25.110 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:09:25.110 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:09:25.110 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:09:25.110 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:09:25.110 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:09:25.111 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:09:25.111 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:09:25.113 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:09:25.113 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:09:25.113 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:09:25.113 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:09:25.113 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:09:25.113 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:09:25.113 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster1 2020-04-10T07:09:25.117 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:09:25.117 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:25.117 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:09:25.118 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:09:25.118 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:09:25.118 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:09:25.118 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:09:25.118 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:09:25.118 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:09:25.118 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:09:25.118 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:09:25.119 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:09:25.119 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:09:25.119 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:09:25.119 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:09:25.129 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:09:25.129 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:09:25.129 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:09:25.129 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test_force_promote 2020-04-10T07:09:25.226 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:Invalid command: unused arguments: ['mirror/test_force_promote'] 2020-04-10T07:09:25.226 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status : get status for rbd mirror 2020-04-10T07:09:25.226 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:09:25.228 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test_force_promote 2020-04-10T07:09:25.229 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:1 2020-04-10T07:09:25.229 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:09:25.229 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:09:25.229 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 cluster instance 2020-04-10T07:09:25.229 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:09:25.229 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:09:25.229 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:09:25.232 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:09:25.232 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:09:25.232 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:09:25.232 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:09:25.232 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:09:25.232 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:09:25.233 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:09:25.233 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:1 cluster1 2020-04-10T07:09:25.233 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:1 2020-04-10T07:09:25.233 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:25.233 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:09:25.233 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2020-04-10T07:09:25.233 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:09:25.234 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:09:25.234 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:09:25.234 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:09:25.234 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:09:25.234 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:09:25.234 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:09:25.234 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:09:25.234 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:09:25.235 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:09:25.235 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2020-04-10T07:09:25.247 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:09:25.247 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:09:25.248 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:09:25.248 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test_force_promote 2020-04-10T07:09:25.337 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:Invalid command: unused arguments: ['mirror/test_force_promote'] 2020-04-10T07:09:25.338 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status : get status for rbd mirror 2020-04-10T07:09:25.338 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:09:25.340 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:2 rbd mirror status mirror/test_force_promote 2020-04-10T07:09:25.340 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:2 2020-04-10T07:09:25.340 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:09:25.340 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:09:25.340 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 cluster instance 2020-04-10T07:09:25.341 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:09:25.341 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:09:25.341 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:09:25.341 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:09:25.341 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:09:25.341 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:09:25.341 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:09:25.341 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:09:25.342 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:09:25.342 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:09:25.342 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:2 cluster1 2020-04-10T07:09:25.342 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:2 2020-04-10T07:09:25.342 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:25.342 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:09:25.342 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2020-04-10T07:09:25.343 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:09:25.343 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:09:25.343 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:09:25.343 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:09:25.343 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:09:25.343 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:09:25.343 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:09:25.343 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:09:25.344 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:09:25.344 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:09:25.344 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2020-04-10T07:09:25.361 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:09:25.361 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:09:25.362 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:09:25.362 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test_force_promote 2020-04-10T07:09:25.451 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:Invalid command: unused arguments: ['mirror/test_force_promote'] 2020-04-10T07:09:25.452 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status : get status for rbd mirror 2020-04-10T07:09:25.452 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:09:25.454 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:3 rbd mirror status mirror/test_force_promote 2020-04-10T07:09:25.454 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:3 2020-04-10T07:09:25.454 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:09:25.454 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:09:25.455 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:3 cluster instance 2020-04-10T07:09:25.455 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:3 2020-04-10T07:09:25.455 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:09:25.455 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:09:25.455 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:09:25.455 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:09:25.455 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 3 = cluster1:3 ] 2020-04-10T07:09:25.456 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:09:25.456 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:09:25.456 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=3 2020-04-10T07:09:25.456 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:09:25.456 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:3 cluster1 2020-04-10T07:09:25.456 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:3 2020-04-10T07:09:25.457 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:25.457 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:09:25.457 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:3 local_cluster instance 2020-04-10T07:09:25.457 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:3 2020-04-10T07:09:25.457 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:09:25.458 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:09:25.458 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:09:25.458 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:09:25.458 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 3 = cluster1:3 ] 2020-04-10T07:09:25.458 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:09:25.458 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:09:25.458 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=3 2020-04-10T07:09:25.459 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:09:25.459 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.3 admin socket 2020-04-10T07:09:25.471 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:09:25.471 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:09:25.471 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:09:25.471 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/test_force_promote 2020-04-10T07:09:25.566 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:09:25.567 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ status_result= "state": "Replaying" 2020-04-10T07:09:25.567 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo "state": "Replaying" 2020-04-10T07:09:25.568 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep -i Replaying 2020-04-10T07:09:25.568 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "state": "Replaying" 2020-04-10T07:09:25.570 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ current_state=started 2020-04-10T07:09:25.570 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test started = started 2020-04-10T07:09:25.571 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:09:25.571 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror test_force_promote 2020-04-10T07:09:25.571 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1 2020-04-10T07:09:25.571 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:25.571 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:25.571 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_force_promote 2020-04-10T07:09:25.571 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ snapshot = journal ] 2020-04-10T07:09:25.572 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ snapshot = snapshot ] 2020-04-10T07:09:25.572 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror test_force_promote 2020-04-10T07:09:25.572 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1 2020-04-10T07:09:25.572 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:25.572 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:25.572 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_force_promote 2020-04-10T07:09:25.573 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster2-mirror-test_force_promote.status 2020-04-10T07:09:25.573 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2-mirror-test_force_promote.status 2020-04-10T07:09:25.573 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:09:25.574 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster2-mirror-test_force_promote.status 2020-04-10T07:09:25.574 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror-test_force_promote.status 2020-04-10T07:09:25.574 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror-test_force_promote.status 2020-04-10T07:09:25.574 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:09:25.575 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_force_promote.status 2020-04-10T07:09:25.575 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mirror_image_snapshot cluster2 mirror test_force_promote 2020-04-10T07:09:25.575 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:25.575 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:25.575 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_force_promote 2020-04-10T07:09:25.575 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test_force_promote 2020-04-10T07:09:26.114 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Snapshot ID: 37 2020-04-10T07:09:26.124 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_newest_mirror_snapshot cluster2 mirror test_force_promote /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test_force_promote.status 2020-04-10T07:09:26.124 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:26.124 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:26.124 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_force_promote 2020-04-10T07:09:26.125 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local log=/tmp/tmp.8SwBCsYPSC/cluster2-mirror-test_force_promote.status 2020-04-10T07:09:26.125 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 snap list --all mirror/test_force_promote --format xml 2020-04-10T07:09:26.125 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2020-04-10T07:09:26.170 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //snapshot/id 2020-04-10T07:09:26.172 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snapshot_id=37 2020-04-10T07:09:26.172 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ true 2020-04-10T07:09:26.173 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0.2 2020-04-10T07:09:26.374 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_newest_mirror_snapshot cluster1 mirror test_force_promote /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_force_promote.status 2020-04-10T07:09:26.374 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:26.374 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:26.375 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_force_promote 2020-04-10T07:09:26.375 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_force_promote.status 2020-04-10T07:09:26.375 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 snap list --all mirror/test_force_promote --format xml 2020-04-10T07:09:26.375 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2020-04-10T07:09:26.445 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2020-04-10T07:09:26.449 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local primary_snapshot_id=36 2020-04-10T07:09:26.449 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test 37 = 36 2020-04-10T07:09:26.449 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0.4 2020-04-10T07:09:26.851 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_newest_mirror_snapshot cluster1 mirror test_force_promote /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_force_promote.status 2020-04-10T07:09:26.851 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:26.851 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:26.851 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_force_promote 2020-04-10T07:09:26.851 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_force_promote.status 2020-04-10T07:09:26.851 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 snap list --all mirror/test_force_promote --format xml 2020-04-10T07:09:26.851 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2020-04-10T07:09:26.917 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2020-04-10T07:09:26.923 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local primary_snapshot_id=36 2020-04-10T07:09:26.924 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test 37 = 36 2020-04-10T07:09:26.924 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0.8 2020-04-10T07:09:27.727 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_newest_mirror_snapshot cluster1 mirror test_force_promote /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_force_promote.status 2020-04-10T07:09:27.728 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:27.728 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:27.728 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_force_promote 2020-04-10T07:09:27.728 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_force_promote.status 2020-04-10T07:09:27.729 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 snap list --all mirror/test_force_promote --format xml 2020-04-10T07:09:27.729 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2020-04-10T07:09:27.794 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2020-04-10T07:09:27.799 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local primary_snapshot_id=37 2020-04-10T07:09:27.799 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test 37 = 37 2020-04-10T07:09:27.799 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:09:27.800 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test_force_promote up+replaying 2020-04-10T07:09:27.800 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:27.800 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:27.800 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_force_promote 2020-04-10T07:09:27.800 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:09:27.800 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:09:27.800 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:09:27.801 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:09:28.800 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_status_in_pool_dir cluster1 mirror test_force_promote up+replaying 2020-04-10T07:09:28.801 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:28.801 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:28.801 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_force_promote 2020-04-10T07:09:28.801 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:09:28.801 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:09:28.801 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:09:28.802 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror-test_force_promote.mirror_status 2020-04-10T07:09:28.802 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror-test_force_promote.mirror_status 2020-04-10T07:09:28.802 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:09:28.802 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_force_promote.mirror_status 2020-04-10T07:09:28.803 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test_force_promote 2020-04-10T07:09:28.803 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ tee /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_force_promote.mirror_status 2020-04-10T07:09:28.873 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:test_force_promote: 2020-04-10T07:09:28.874 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: global_id: b192e243-6abc-405a-b798-b60b9d74691e 2020-04-10T07:09:28.874 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+replaying 2020-04-10T07:09:28.874 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: replaying, {"replay_state": "idle", "remote_snapshot_timestamp": 1586502561} 2020-04-10T07:09:28.874 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: service: mirror.3 on smithi204 2020-04-10T07:09:28.874 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:09:23 2020-04-10T07:09:28.874 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: peer_sites: 2020-04-10T07:09:28.875 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: name: cluster2-DNE 2020-04-10T07:09:28.875 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+stopped 2020-04-10T07:09:28.875 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: local image is primary 2020-04-10T07:09:28.875 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:09:26 2020-04-10T07:09:28.882 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ state: .*up+replaying /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_force_promote.mirror_status 2020-04-10T07:09:28.883 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+replaying 2020-04-10T07:09:28.884 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ description: .* /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_force_promote.mirror_status 2020-04-10T07:09:28.886 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: replaying, {"replay_state": "idle", "remote_snapshot_timestamp": 1586502561} 2020-04-10T07:09:28.886 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:09:28.887 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+replaying 2020-04-10T07:09:28.887 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^up+ 2020-04-10T07:09:28.887 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+replaying 2020-04-10T07:09:28.887 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep service: *mirror..* on /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_force_promote.mirror_status 2020-04-10T07:09:28.888 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: service: mirror.3 on smithi204 2020-04-10T07:09:28.889 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_force_promote.mirror_status 2020-04-10T07:09:28.891 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update=2020-04-10 07:09:23 2020-04-10T07:09:28.891 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test_force_promote up+replaying 2020-04-10 07:09:23 2020-04-10T07:09:28.891 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:28.892 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:28.892 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_force_promote 2020-04-10T07:09:28.892 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:09:28.892 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local prev_last_update=2020-04-10 07:09:23 2020-04-10T07:09:28.892 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror.mirror_status 2020-04-10T07:09:28.893 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror.mirror_status 2020-04-10T07:09:28.893 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:09:28.894 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror.mirror_status 2020-04-10T07:09:28.894 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2020-04-10T07:09:29.140 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: failed to query services: (13) Permission denied 2020-04-10T07:09:29.192 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update state 2020-04-10T07:09:29.192 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test_force_promote']/last_update 2020-04-10T07:09:29.198 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ last_update=2020-04-10 07:09:23 2020-04-10T07:09:29.198 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test_force_promote']/state 2020-04-10T07:09:29.202 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ state=up+replaying 2020-04-10T07:09:29.203 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+replaying 2020-04-10T07:09:29.203 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep up+replaying 2020-04-10T07:09:29.204 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+replaying 2020-04-10T07:09:29.205 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:09:29.205 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:09:29.205 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_status_in_pool_dir cluster2 mirror test_force_promote up+stopped 2020-04-10T07:09:29.205 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:29.205 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:29.205 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_force_promote 2020-04-10T07:09:29.206 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+stopped 2020-04-10T07:09:29.206 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:09:29.206 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:09:29.206 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:09:30.207 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_status_in_pool_dir cluster2 mirror test_force_promote up+stopped 2020-04-10T07:09:30.207 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:30.207 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:30.207 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_force_promote 2020-04-10T07:09:30.207 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+stopped 2020-04-10T07:09:30.208 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:09:30.208 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:09:30.208 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster2-mirror-test_force_promote.mirror_status 2020-04-10T07:09:30.208 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2-mirror-test_force_promote.mirror_status 2020-04-10T07:09:30.208 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:09:30.209 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster2-mirror-test_force_promote.mirror_status 2020-04-10T07:09:30.209 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ + CEPH_ARGS= rbd --cluster cluster2 mirrortee image /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test_force_promote.mirror_status status 2020-04-10T07:09:30.209 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: mirror/test_force_promote 2020-04-10T07:09:30.275 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:test_force_promote: 2020-04-10T07:09:30.276 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: global_id: b192e243-6abc-405a-b798-b60b9d74691e 2020-04-10T07:09:30.276 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+stopped 2020-04-10T07:09:30.276 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: local image is primary 2020-04-10T07:09:30.276 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: service: mirror.2 on smithi204 2020-04-10T07:09:30.276 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:09:26 2020-04-10T07:09:30.276 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: peer_sites: 2020-04-10T07:09:30.277 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: name: cluster1-DNE 2020-04-10T07:09:30.277 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+replaying 2020-04-10T07:09:30.277 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: replaying, {"replay_state": "idle", "remote_snapshot_timestamp": 1586502561} 2020-04-10T07:09:30.277 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:09:23 2020-04-10T07:09:30.277 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: snapshots: 2020-04-10T07:09:30.278 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: 37 .mirror.primary.b192e243-6abc-405a-b798-b60b9d74691e.0db2d7ca-a5c4-4345-8bfe-5be936127d5b (peer_uuids:[90b6cfa2-5481-4edf-b462-5494aab62f6f]) 2020-04-10T07:09:30.287 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ state: .*up+stopped /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test_force_promote.mirror_status 2020-04-10T07:09:30.287 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ description: .* /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test_force_promote.mirror_status 2020-04-10T07:09:30.287 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:09:30.287 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+stopped 2020-04-10T07:09:30.287 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^up+ 2020-04-10T07:09:30.288 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep service: *mirror..* on /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test_force_promote.mirror_status 2020-04-10T07:09:30.288 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test_force_promote.mirror_status 2020-04-10T07:09:30.288 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+stopped 2020-04-10T07:09:30.288 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: local image is primary 2020-04-10T07:09:30.288 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+stopped 2020-04-10T07:09:30.289 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: service: mirror.2 on smithi204 2020-04-10T07:09:30.290 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update=2020-04-10 07:09:26 2020-04-10T07:09:30.290 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_mirror_pool_status_verbose cluster2 mirror test_force_promote up+stopped 2020-04-10 07:09:26 2020-04-10T07:09:30.290 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:30.291 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:30.291 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_force_promote 2020-04-10T07:09:30.291 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+stopped 2020-04-10T07:09:30.291 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local prev_last_update=2020-04-10 07:09:26 2020-04-10T07:09:30.291 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster2-mirror.mirror_status 2020-04-10T07:09:30.292 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2-mirror.mirror_status 2020-04-10T07:09:30.292 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:09:30.292 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster2-mirror.mirror_status 2020-04-10T07:09:30.292 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2020-04-10T07:09:30.331 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: failed to query services: (13) Permission denied 2020-04-10T07:09:30.373 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update state 2020-04-10T07:09:30.373 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test_force_promote']/last_update 2020-04-10T07:09:30.376 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ last_update=2020-04-10 07:09:26 2020-04-10T07:09:30.376 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test_force_promote']/state 2020-04-10T07:09:30.379 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ state=up+stopped 2020-04-10T07:09:30.379 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+stopped 2020-04-10T07:09:30.379 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep up+stopped 2020-04-10T07:09:30.380 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+stopped 2020-04-10T07:09:30.380 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:09:30.381 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:09:30.381 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ promote_image cluster1 mirror test_force_promote --force 2020-04-10T07:09:30.381 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:30.381 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:30.381 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_force_promote 2020-04-10T07:09:30.381 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local force=--force 2020-04-10T07:09:30.381 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster1 mirror image promote mirror/test_force_promote --force 2020-04-10T07:09:37.112 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Image promoted to primary 2020-04-10T07:09:37.121 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_stopped cluster1 mirror test_force_promote 2020-04-10T07:09:37.122 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:37.122 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:37.122 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_force_promote 2020-04-10T07:09:37.122 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_state cluster1 mirror test_force_promote stopped 2020-04-10T07:09:37.122 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:37.122 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:37.123 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_force_promote 2020-04-10T07:09:37.123 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state=stopped 2020-04-10T07:09:37.123 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local s 2020-04-10T07:09:37.123 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:09:38.134 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_image_replay_state cluster1 mirror test_force_promote stopped 2020-04-10T07:09:38.135 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:38.135 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:38.135 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_force_promote 2020-04-10T07:09:38.135 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local test_state=stopped 2020-04-10T07:09:38.135 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_result 2020-04-10T07:09:38.136 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_state=stopped 2020-04-10T07:09:38.136 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test_force_promote 2020-04-10T07:09:38.136 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_instance=cluster1 2020-04-10T07:09:38.136 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ + shift 2020-04-10T07:09:38.136 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:38.137 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:grep+ local -i instance=cluster1 2020-04-10T07:09:38.137 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state 2020-04-10T07:09:38.137 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local loop_instance 2020-04-10T07:09:38.137 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0 2020-04-10T07:09:38.137 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ cluster1 != cluster1 ] 2020-04-10T07:09:38.138 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:09:38.138 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test_force_promote 2020-04-10T07:09:38.138 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:09:38.138 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:09:38.138 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:09:38.139 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:09:38.139 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:09:38.139 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:09:38.139 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:09:38.139 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:09:38.140 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:09:38.140 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:09:38.140 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:09:38.140 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:09:38.140 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:09:38.141 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:09:38.141 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster1 2020-04-10T07:09:38.141 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:09:38.142 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:38.142 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:09:38.142 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:09:38.142 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:09:38.142 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:09:38.143 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:09:38.143 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:09:38.143 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:09:38.143 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:09:38.144 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:09:38.144 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:09:38.144 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:09:38.145 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:09:38.145 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:09:38.151 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:09:38.151 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:09:38.151 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:09:38.152 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test_force_promote 2020-04-10T07:09:38.247 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:Invalid command: unused arguments: ['mirror/test_force_promote'] 2020-04-10T07:09:38.247 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status : get status for rbd mirror 2020-04-10T07:09:38.247 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:09:38.249 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test_force_promote 2020-04-10T07:09:38.249 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:1 2020-04-10T07:09:38.249 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:09:38.250 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:09:38.250 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 cluster instance 2020-04-10T07:09:38.250 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:09:38.250 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:09:38.250 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:09:38.250 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:09:38.250 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:09:38.251 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:09:38.251 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:09:38.251 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:09:38.251 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:09:38.251 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:09:38.251 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:1 cluster1 2020-04-10T07:09:38.258 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:1 2020-04-10T07:09:38.258 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:38.258 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:09:38.258 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2020-04-10T07:09:38.259 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:09:38.259 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:09:38.259 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:09:38.259 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:09:38.259 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:09:38.259 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:09:38.259 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:09:38.259 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:09:38.260 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:09:38.260 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:09:38.260 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2020-04-10T07:09:38.273 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:09:38.273 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:09:38.273 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:09:38.273 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test_force_promote 2020-04-10T07:09:38.364 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:Invalid command: unused arguments: ['mirror/test_force_promote'] 2020-04-10T07:09:38.364 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status : get status for rbd mirror 2020-04-10T07:09:38.364 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:09:38.367 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:2 rbd mirror status mirror/test_force_promote 2020-04-10T07:09:38.367 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:2 2020-04-10T07:09:38.367 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:09:38.367 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:09:38.367 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 cluster instance 2020-04-10T07:09:38.367 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:09:38.368 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:09:38.368 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:09:38.368 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:09:38.368 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:09:38.368 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:09:38.368 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:09:38.368 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:09:38.368 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:09:38.369 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:09:38.369 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:2 cluster1 2020-04-10T07:09:38.369 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:2 2020-04-10T07:09:38.369 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:38.369 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:09:38.369 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2020-04-10T07:09:38.369 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:09:38.370 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:09:38.370 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:09:38.370 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:09:38.370 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:09:38.370 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:09:38.370 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:09:38.370 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:09:38.370 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:09:38.371 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:09:38.371 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2020-04-10T07:09:38.388 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:09:38.388 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:09:38.389 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:09:38.389 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test_force_promote 2020-04-10T07:09:38.477 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:Invalid command: unused arguments: ['mirror/test_force_promote'] 2020-04-10T07:09:38.477 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status : get status for rbd mirror 2020-04-10T07:09:38.478 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:09:38.480 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:3 rbd mirror status mirror/test_force_promote 2020-04-10T07:09:38.480 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:3 2020-04-10T07:09:38.480 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:09:38.480 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:09:38.480 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:3 cluster instance 2020-04-10T07:09:38.481 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:3 2020-04-10T07:09:38.481 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:09:38.481 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:09:38.481 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:09:38.481 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:09:38.481 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 3 = cluster1:3 ] 2020-04-10T07:09:38.481 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:09:38.481 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:09:38.482 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=3 2020-04-10T07:09:38.482 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:09:38.482 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:3 cluster1 2020-04-10T07:09:38.482 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:3 2020-04-10T07:09:38.482 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:38.482 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:09:38.482 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:3 local_cluster instance 2020-04-10T07:09:38.483 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:3 2020-04-10T07:09:38.483 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:09:38.483 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:09:38.483 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:09:38.483 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:09:38.483 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 3 = cluster1:3 ] 2020-04-10T07:09:38.483 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:09:38.483 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:09:38.484 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=3 2020-04-10T07:09:38.484 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:09:38.484 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.3 admin socket 2020-04-10T07:09:38.498 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:09:38.498 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:09:38.498 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:09:38.498 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/test_force_promote 2020-04-10T07:09:38.596 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:09:38.597 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ status_result= "state": "Stopped" 2020-04-10T07:09:38.598 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo "state": "Stopped" 2020-04-10T07:09:38.598 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep -i Replaying 2020-04-10T07:09:38.598 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test stopped = stopped 2020-04-10T07:09:38.599 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:09:38.599 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_stopped cluster2 mirror test_force_promote 2020-04-10T07:09:38.599 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:38.599 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:38.599 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_force_promote 2020-04-10T07:09:38.599 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_state cluster2 mirror test_force_promote stopped 2020-04-10T07:09:38.599 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:38.599 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:38.600 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_force_promote 2020-04-10T07:09:38.600 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state=stopped 2020-04-10T07:09:38.600 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local s 2020-04-10T07:09:38.600 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:09:39.599 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_image_replay_state cluster2 mirror test_force_promote stopped 2020-04-10T07:09:39.599 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:39.599 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:39.600 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_force_promote 2020-04-10T07:09:39.600 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local test_state=stopped 2020-04-10T07:09:39.600 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_result 2020-04-10T07:09:39.600 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_state=stopped 2020-04-10T07:09:39.601 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemons cluster2 rbd mirror status mirror/test_force_promote 2020-04-10T07:09:39.601 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local+ cluster_instance=cluster2 2020-04-10T07:09:39.601 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:09:39.602 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:39.602 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ greplocal instance=cluster2 -i 2020-04-10T07:09:39.602 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state+ 2020-04-10T07:09:39.602 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:local loop_instance 2020-04-10T07:09:39.602 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0 2020-04-10T07:09:39.602 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ cluster2 != cluster2 ] 2020-04-10T07:09:39.602 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:09:39.604 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster2:0 rbd mirror status mirror/test_force_promote 2020-04-10T07:09:39.604 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2:0 2020-04-10T07:09:39.604 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:09:39.604 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:09:39.605 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:0 cluster instance 2020-04-10T07:09:39.605 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:0 2020-04-10T07:09:39.605 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:09:39.605 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:09:39.605 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:09:39.605 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:09:39.605 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster2:0 ] 2020-04-10T07:09:39.606 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster2 2020-04-10T07:09:39.606 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:09:39.606 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:09:39.606 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:09:39.606 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster2:0 cluster2 2020-04-10T07:09:39.606 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster2:0 2020-04-10T07:09:39.606 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:39.606 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:09:39.607 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:0 local_cluster instance 2020-04-10T07:09:39.607 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:0 2020-04-10T07:09:39.607 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:09:39.607 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:09:39.607 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:09:39.615 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:09:39.615 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster2:0 ] 2020-04-10T07:09:39.615 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster2 2020-04-10T07:09:39.615 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster2 2020-04-10T07:09:39.616 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:09:39.616 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:09:39.616 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster2 --name client.mirror.0 admin socket 2020-04-10T07:09:39.624 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster2-client.mirror.0.asok 2020-04-10T07:09:39.625 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster2-client.mirror.0.asok 2020-04-10T07:09:39.625 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster2-client.mirror.0.asok 2020-04-10T07:09:39.625 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster2-client.mirror.0.asok rbd mirror status mirror/test_force_promote 2020-04-10T07:09:39.713 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:Invalid command: unused arguments: ['mirror/test_force_promote'] 2020-04-10T07:09:39.714 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status : get status for rbd mirror 2020-04-10T07:09:39.714 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:09:39.716 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster2:1 rbd mirror status mirror/test_force_promote 2020-04-10T07:09:39.716 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2:1 2020-04-10T07:09:39.716 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:09:39.716 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:09:39.716 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:1 cluster instance 2020-04-10T07:09:39.716 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:1 2020-04-10T07:09:39.717 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:09:39.717 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:09:39.717 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:09:39.717 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:09:39.717 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster2:1 ] 2020-04-10T07:09:39.717 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster2 2020-04-10T07:09:39.717 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:09:39.718 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:09:39.718 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:09:39.718 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster2:1 cluster2 2020-04-10T07:09:39.718 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster2:1 2020-04-10T07:09:39.718 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:39.718 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:09:39.718 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:1 local_cluster instance 2020-04-10T07:09:39.718 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:1 2020-04-10T07:09:39.719 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:09:39.719 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:09:39.719 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:09:39.719 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:09:39.719 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster2:1 ] 2020-04-10T07:09:39.719 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster2 2020-04-10T07:09:39.719 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster2 2020-04-10T07:09:39.720 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:09:39.720 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:09:39.720 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster2 --name client.mirror.1 admin socket 2020-04-10T07:09:39.737 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster2-client.mirror.1.asok 2020-04-10T07:09:39.738 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster2-client.mirror.1.asok 2020-04-10T07:09:39.738 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster2-client.mirror.1.asok 2020-04-10T07:09:39.738 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster2-client.mirror.1.asok rbd mirror status mirror/test_force_promote 2020-04-10T07:09:39.832 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:Invalid command: unused arguments: ['mirror/test_force_promote'] 2020-04-10T07:09:39.832 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status : get status for rbd mirror 2020-04-10T07:09:39.832 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:09:39.832 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster2:2 rbd mirror status mirror/test_force_promote 2020-04-10T07:09:39.832 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2:2 2020-04-10T07:09:39.832 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:09:39.833 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:09:39.833 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:2 cluster instance 2020-04-10T07:09:39.833 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:2 2020-04-10T07:09:39.833 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:09:39.833 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:09:39.833 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:09:39.834 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:09:39.834 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster2:2 ] 2020-04-10T07:09:39.834 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster2 2020-04-10T07:09:39.834 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:09:39.834 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:09:39.834 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:09:39.834 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster2:2 cluster2 2020-04-10T07:09:39.834 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster2:2 2020-04-10T07:09:39.835 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:39.835 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:09:39.835 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:2 local_cluster instance 2020-04-10T07:09:39.835 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:2 2020-04-10T07:09:39.835 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:09:39.835 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:09:39.835 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:09:39.836 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:09:39.836 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster2:2 ] 2020-04-10T07:09:39.836 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster2 2020-04-10T07:09:39.836 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster2 2020-04-10T07:09:39.836 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:09:39.836 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:09:39.838 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster2 --name client.mirror.2 admin socket 2020-04-10T07:09:39.851 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster2-client.mirror.2.asok 2020-04-10T07:09:39.851 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster2-client.mirror.2.asok 2020-04-10T07:09:39.851 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster2-client.mirror.2.asok 2020-04-10T07:09:39.851 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster2-client.mirror.2.asok rbd mirror status mirror/test_force_promote 2020-04-10T07:09:39.955 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:09:39.956 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ status_result= "state": "Stopped" 2020-04-10T07:09:39.956 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo "state": "Stopped" 2020-04-10T07:09:39.956 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep -i Replaying 2020-04-10T07:09:39.956 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test stopped = stopped 2020-04-10T07:09:39.956 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:09:39.956 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test_force_promote up+stopped 2020-04-10T07:09:39.957 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:39.957 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:39.957 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_force_promote 2020-04-10T07:09:39.957 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+stopped 2020-04-10T07:09:39.957 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:09:39.957 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:09:39.957 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:09:40.950 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_status_in_pool_dir cluster1 mirror test_force_promote up+stopped 2020-04-10T07:09:40.950 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:40.950 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:40.951 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_force_promote 2020-04-10T07:09:40.951 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+stopped 2020-04-10T07:09:40.951 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:09:40.951 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:09:40.951 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror-test_force_promote.mirror_status 2020-04-10T07:09:40.952 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror-test_force_promote.mirror_status 2020-04-10T07:09:40.952 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:09:40.953 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_force_promote.mirror_status 2020-04-10T07:09:40.953 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test_force_promote 2020-04-10T07:09:40.953 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ tee /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_force_promote.mirror_status 2020-04-10T07:09:41.033 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:test_force_promote: 2020-04-10T07:09:41.033 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: global_id: b192e243-6abc-405a-b798-b60b9d74691e 2020-04-10T07:09:41.034 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+stopped 2020-04-10T07:09:41.034 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: local image is primary 2020-04-10T07:09:41.034 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: service: mirror.3 on smithi204 2020-04-10T07:09:41.034 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:09:39 2020-04-10T07:09:41.035 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: peer_sites: 2020-04-10T07:09:41.035 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: name: cluster2-DNE 2020-04-10T07:09:41.035 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+stopped 2020-04-10T07:09:41.035 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: local image is primary 2020-04-10T07:09:41.035 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:09:36 2020-04-10T07:09:41.036 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: snapshots: 2020-04-10T07:09:41.036 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: 36 .mirror.primary.b192e243-6abc-405a-b798-b60b9d74691e.d7da4eb0-9ce3-48d5-9c9c-a1cc499f7864 (peer_uuids:[2571f49c-d767-4112-ae12-d8b71263d58b]) 2020-04-10T07:09:41.037 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ state: .*up+stopped /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_force_promote.mirror_status 2020-04-10T07:09:41.037 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ description: .* /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_force_promote.mirror_status 2020-04-10T07:09:41.037 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+stopped 2020-04-10T07:09:41.039 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: local image is primary 2020-04-10T07:09:41.039 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:09:41.039 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+stopped 2020-04-10T07:09:41.039 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^up+ 2020-04-10T07:09:41.040 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+stopped 2020-04-10T07:09:41.040 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep service: *mirror..* on /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_force_promote.mirror_status 2020-04-10T07:09:41.041 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: service: mirror.3 on smithi204 2020-04-10T07:09:41.041 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_force_promote.mirror_status 2020-04-10T07:09:41.043 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update=2020-04-10 07:09:39 2020-04-10T07:09:41.043 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test_force_promote up+stopped 2020-04-10 07:09:39 2020-04-10T07:09:41.043 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:41.044 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:41.044 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_force_promote 2020-04-10T07:09:41.044 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+stopped 2020-04-10T07:09:41.044 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local prev_last_update=2020-04-10 07:09:39 2020-04-10T07:09:41.044 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror.mirror_status 2020-04-10T07:09:41.044 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror.mirror_status 2020-04-10T07:09:41.044 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:09:41.045 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror.mirror_status 2020-04-10T07:09:41.045 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2020-04-10T07:09:41.100 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: failed to query services: (13) Permission denied 2020-04-10T07:09:41.140 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update state 2020-04-10T07:09:41.140 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test_force_promote']/last_update 2020-04-10T07:09:41.146 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ last_update=2020-04-10 07:09:39 2020-04-10T07:09:41.146 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test_force_promote']/state 2020-04-10T07:09:41.151 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ state=up+stopped 2020-04-10T07:09:41.151 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+stopped 2020-04-10T07:09:41.151 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep up+stopped 2020-04-10T07:09:41.153 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+stopped 2020-04-10T07:09:41.153 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:09:41.153 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:09:41.153 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_status_in_pool_dir cluster2 mirror test_force_promote up+stopped 2020-04-10T07:09:41.154 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:41.154 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:41.154 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_force_promote 2020-04-10T07:09:41.154 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+stopped 2020-04-10T07:09:41.154 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:09:41.155 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:09:41.155 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:09:42.155 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_status_in_pool_dir cluster2 mirror test_force_promote up+stopped 2020-04-10T07:09:42.155 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:42.155 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:42.155 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_force_promote 2020-04-10T07:09:42.155 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+stopped 2020-04-10T07:09:42.156 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:09:42.156 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:09:42.156 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster2-mirror-test_force_promote.mirror_status 2020-04-10T07:09:42.156 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2-mirror-test_force_promote.mirror_status 2020-04-10T07:09:42.156 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:09:42.157 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster2-mirror-test_force_promote.mirror_status 2020-04-10T07:09:42.157 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster2 mirror image status mirror/test_force_promote 2020-04-10T07:09:42.157 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ tee /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test_force_promote.mirror_status 2020-04-10T07:09:42.223 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:test_force_promote: 2020-04-10T07:09:42.224 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: global_id: b192e243-6abc-405a-b798-b60b9d74691e 2020-04-10T07:09:42.224 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+stopped 2020-04-10T07:09:42.224 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: local image is primary 2020-04-10T07:09:42.224 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: service: mirror.2 on smithi204 2020-04-10T07:09:42.224 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:09:41 2020-04-10T07:09:42.224 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: peer_sites: 2020-04-10T07:09:42.224 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: name: cluster1-DNE 2020-04-10T07:09:42.225 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+stopped 2020-04-10T07:09:42.225 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: local image is primary 2020-04-10T07:09:42.225 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:09:39 2020-04-10T07:09:42.225 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: snapshots: 2020-04-10T07:09:42.225 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: 37 .mirror.primary.b192e243-6abc-405a-b798-b60b9d74691e.0db2d7ca-a5c4-4345-8bfe-5be936127d5b (peer_uuids:[90b6cfa2-5481-4edf-b462-5494aab62f6f]) 2020-04-10T07:09:42.229 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ state: .*up+stopped /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test_force_promote.mirror_status 2020-04-10T07:09:42.230 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+stopped 2020-04-10T07:09:42.230 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ description: .* /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test_force_promote.mirror_status 2020-04-10T07:09:42.231 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: local image is primary 2020-04-10T07:09:42.231 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:09:42.232 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+stopped 2020-04-10T07:09:42.232 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^up+ 2020-04-10T07:09:42.232 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+stopped 2020-04-10T07:09:42.233 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep service: *mirror..* on /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test_force_promote.mirror_status 2020-04-10T07:09:42.233 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: service: mirror.2 on smithi204 2020-04-10T07:09:42.234 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test_force_promote.mirror_status 2020-04-10T07:09:42.235 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update=2020-04-10 07:09:41 2020-04-10T07:09:42.235 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_mirror_pool_status_verbose cluster2 mirror test_force_promote up+stopped 2020-04-10 07:09:41 2020-04-10T07:09:42.236 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:42.236 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:42.236 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_force_promote 2020-04-10T07:09:42.236 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+stopped 2020-04-10T07:09:42.236 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local prev_last_update=2020-04-10 07:09:41 2020-04-10T07:09:42.236 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster2-mirror.mirror_status 2020-04-10T07:09:42.236 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2-mirror.mirror_status 2020-04-10T07:09:42.237 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:09:42.237 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster2-mirror.mirror_status 2020-04-10T07:09:42.237 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2020-04-10T07:09:42.273 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: failed to query services: (13) Permission denied 2020-04-10T07:09:42.313 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update state 2020-04-10T07:09:42.313 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test_force_promote']/last_update 2020-04-10T07:09:42.316 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ last_update=2020-04-10 07:09:41 2020-04-10T07:09:42.316 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test_force_promote']/state 2020-04-10T07:09:42.318 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ state=up+stopped 2020-04-10T07:09:42.318 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+stopped 2020-04-10T07:09:42.318 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep up+stopped 2020-04-10T07:09:42.320 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+stopped 2020-04-10T07:09:42.320 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:09:42.320 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:09:42.320 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ write_image cluster1 mirror test_force_promote 100 2020-04-10T07:09:42.320 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:42.320 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:42.321 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_force_promote 2020-04-10T07:09:42.321 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local count=100 2020-04-10T07:09:42.321 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local size= 2020-04-10T07:09:42.321 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 2020-04-10T07:09:42.321 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ size=4096 2020-04-10T07:09:42.321 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 bench mirror/test_force_promote --io-type write --io-size 4096 --io-threads 1 --io-total 409600 --io-pattern rand 2020-04-10T07:09:42.406 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:bench type write io_size 4096 io_threads 1 bytes 409600 pattern random 2020-04-10T07:09:42.476 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: SEC OPS OPS/SEC BYTES/SEC 2020-04-10T07:09:42.477 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:elapsed: 0 ops: 100 ops/sec: 1388.89 bytes/sec: 5.4 MiB/s 2020-04-10T07:09:42.493 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ write_image cluster2 mirror test_force_promote 100 2020-04-10T07:09:42.493 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:42.493 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:42.494 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_force_promote 2020-04-10T07:09:42.494 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local count=100 2020-04-10T07:09:42.494 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local size= 2020-04-10T07:09:42.494 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 2020-04-10T07:09:42.494 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ size=4096 2020-04-10T07:09:42.494 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 bench mirror/test_force_promote --io-type write --io-size 4096 --io-threads 1 --io-total 409600 --io-pattern rand 2020-04-10T07:09:42.579 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:bench type write io_size 4096 io_threads 1 bytes 409600 pattern random 2020-04-10T07:09:42.651 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: SEC OPS OPS/SEC BYTES/SEC 2020-04-10T07:09:42.651 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:elapsed: 0 ops: 100 ops/sec: 1388.89 bytes/sec: 5.4 MiB/s 2020-04-10T07:09:42.667 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ testlog TEST: cloned images 2020-04-10T07:09:42.667 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ tee -a /tmp/tmp.8SwBCsYPSC/rbd-mirror.test.log 2020-04-10T07:09:42.667 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ date +%F %T 2020-04-10T07:09:42.667 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10 07:09:42 TEST: cloned images 2020-04-10T07:09:42.668 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:2020-04-10 07:09:42 TEST: cloned images 2020-04-10T07:09:42.668 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ testlog - default 2020-04-10T07:09:42.668 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ tee -a /tmp/tmp.8SwBCsYPSC/rbd-mirror.test.log 2020-04-10T07:09:42.668 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ date +%F %T 2020-04-10T07:09:42.668 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10 07:09:42 - default 2020-04-10T07:09:42.669 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:2020-04-10 07:09:42 - default 2020-04-10T07:09:42.669 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ parent_image=test_parent 2020-04-10T07:09:42.669 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ parent_snap=snap 2020-04-10T07:09:42.669 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ create_image_and_enable_mirror cluster2 mirror_parent test_parent 2020-04-10T07:09:42.669 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:42.669 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:09:42.670 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror_parent 2020-04-10T07:09:42.670 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:09:42.670 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_parent 2020-04-10T07:09:42.670 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:09:42.670 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local mode=snapshot 2020-04-10T07:09:42.670 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:09:42.670 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ create_image cluster2 mirror_parent test_parent 2020-04-10T07:09:42.671 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:42.671 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:09:42.671 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror_parent 2020-04-10T07:09:42.671 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:09:42.671 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_parent 2020-04-10T07:09:42.671 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:09:42.672 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local size=128 2020-04-10T07:09:42.672 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:09:42.672 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 create --size 128 --image-feature layering,exclusive-lock,journaling mirror_parent/test_parent 2020-04-10T07:09:42.719 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ enable_mirror cluster2 mirror_parent test_parent snapshot 2020-04-10T07:09:42.719 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:42.719 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror_parent 2020-04-10T07:09:42.719 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_parent 2020-04-10T07:09:42.719 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local mode=snapshot 2020-04-10T07:09:42.719 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster2 mirror image enable mirror_parent/test_parent snapshot 2020-04-10T07:09:43.137 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Mirroring enabled 2020-04-10T07:09:43.145 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ write_image cluster2 mirror_parent test_parent 100 2020-04-10T07:09:43.146 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:43.146 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror_parent 2020-04-10T07:09:43.146 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_parent 2020-04-10T07:09:43.146 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local count=100 2020-04-10T07:09:43.146 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local size= 2020-04-10T07:09:43.146 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 2020-04-10T07:09:43.147 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ size=4096 2020-04-10T07:09:43.147 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 bench mirror_parent/test_parent --io-type write --io-size 4096 --io-threads 1 --io-total 409600 --io-pattern rand 2020-04-10T07:09:43.214 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:bench type write io_size 4096 io_threads 1 bytes 409600 pattern random 2020-04-10T07:09:43.276 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: SEC OPS OPS/SEC BYTES/SEC 2020-04-10T07:09:43.276 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:elapsed: 0 ops: 100 ops/sec: 1666.67 bytes/sec: 6.5 MiB/s 2020-04-10T07:09:43.286 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ create_snapshot cluster2 mirror_parent test_parent snap 2020-04-10T07:09:43.286 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:43.286 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror_parent 2020-04-10T07:09:43.287 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_parent 2020-04-10T07:09:43.287 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snap=snap 2020-04-10T07:09:43.287 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 snap create mirror_parent/test_parent@snap 2020-04-10T07:09:44.139 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ protect_snapshot cluster2 mirror_parent test_parent snap 2020-04-10T07:09:44.139 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:44.139 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror_parent 2020-04-10T07:09:44.139 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_parent 2020-04-10T07:09:44.139 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snap=snap 2020-04-10T07:09:44.140 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 snap protect mirror_parent/test_parent@snap 2020-04-10T07:09:44.210 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ clone_image=test_clone 2020-04-10T07:09:44.210 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ clone_image cluster2 mirror_parent test_parent snap mirror test_clone 2020-04-10T07:09:44.210 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:44.211 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local parent_pool=mirror_parent 2020-04-10T07:09:44.211 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local parent_image=test_parent 2020-04-10T07:09:44.211 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local parent_snap=snap 2020-04-10T07:09:44.211 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local clone_pool=mirror 2020-04-10T07:09:44.211 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local clone_image=test_clone 2020-04-10T07:09:44.212 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 6 2020-04-10T07:09:44.212 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 clone mirror_parent/test_parent@snap mirror/test_clone --image-feature layering,exclusive-lock,journaling 2020-04-10T07:09:44.312 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ write_image cluster2 mirror test_clone 100 2020-04-10T07:09:44.313 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:44.313 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:44.313 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_clone 2020-04-10T07:09:44.313 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local count=100 2020-04-10T07:09:44.313 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local size= 2020-04-10T07:09:44.313 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 2020-04-10T07:09:44.313 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ size=4096 2020-04-10T07:09:44.314 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 bench mirror/test_clone --io-type write --io-size 4096 --io-threads 1 --io-total 409600 --io-pattern rand 2020-04-10T07:09:44.404 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:bench type write io_size 4096 io_threads 1 bytes 409600 pattern random 2020-04-10T07:09:44.503 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: SEC OPS OPS/SEC BYTES/SEC 2020-04-10T07:09:44.503 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:elapsed: 0 ops: 100 ops/sec: 1000 bytes/sec: 3.9 MiB/s 2020-04-10T07:09:44.514 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ enable_mirror cluster2 mirror test_clone snapshot 2020-04-10T07:09:44.514 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:44.514 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:44.515 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_clone 2020-04-10T07:09:44.515 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local mode=snapshot 2020-04-10T07:09:44.515 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/test_clone snapshot 2020-04-10T07:09:45.155 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Mirroring enabled 2020-04-10T07:09:45.166 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_started cluster1 mirror_parent test_parent 2020-04-10T07:09:45.166 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:45.166 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror_parent 2020-04-10T07:09:45.166 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_parent 2020-04-10T07:09:45.167 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_state cluster1 mirror_parent test_parent started 2020-04-10T07:09:45.167 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:45.167 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror_parent 2020-04-10T07:09:45.167 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_parent 2020-04-10T07:09:45.167 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state=started 2020-04-10T07:09:45.167 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local s 2020-04-10T07:09:45.167 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:09:46.167 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_image_replay_state cluster1 mirror_parent test_parent started 2020-04-10T07:09:46.167 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:46.167 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror_parent 2020-04-10T07:09:46.167 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_parent 2020-04-10T07:09:46.167 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local test_state=started 2020-04-10T07:09:46.168 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_result 2020-04-10T07:09:46.168 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_state=stopped 2020-04-10T07:09:46.169 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemons cluster1 rbd mirror status mirror_parent/test_parent 2020-04-10T07:09:46.169 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_instance=cluster1 2020-04-10T07:09:46.169 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:09:46.169 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:46.169 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance=cluster1 2020-04-10T07:09:46.169 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local loop_instance 2020-04-10T07:09:46.170 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0 2020-04-10T07:09:46.170 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep -i state 2020-04-10T07:09:46.170 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ cluster1 != cluster1 ] 2020-04-10T07:09:46.172 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:09:46.172 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror_parent/test_parent 2020-04-10T07:09:46.172 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:09:46.172 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:09:46.172 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:09:46.172 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:09:46.173 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:09:46.173 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:09:46.174 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:09:46.174 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:09:46.174 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:09:46.174 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:09:46.174 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:09:46.174 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:09:46.174 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:09:46.175 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:09:46.175 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster1 2020-04-10T07:09:46.175 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:09:46.175 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:46.175 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:09:46.175 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:09:46.175 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:09:46.177 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:09:46.177 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:09:46.177 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:09:46.177 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:09:46.178 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:09:46.178 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:09:46.178 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:09:46.178 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:09:46.178 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:09:46.178 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:09:46.192 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:09:46.192 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:09:46.192 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:09:46.192 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror_parent/test_parent 2020-04-10T07:09:46.298 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:09:46.299 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ status_result= "state": "Replaying" 2020-04-10T07:09:46.299 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep -i Replaying 2020-04-10T07:09:46.299 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo "state": "Replaying" 2020-04-10T07:09:46.301 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "state": "Replaying" 2020-04-10T07:09:46.301 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ current_state=started 2020-04-10T07:09:46.302 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test started = started 2020-04-10T07:09:46.302 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:09:46.302 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror_parent test_parent 2020-04-10T07:09:46.302 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1 2020-04-10T07:09:46.302 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:46.302 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror_parent 2020-04-10T07:09:46.302 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_parent 2020-04-10T07:09:46.303 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ snapshot = journal ] 2020-04-10T07:09:46.303 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ snapshot = snapshot ] 2020-04-10T07:09:46.303 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror_parent test_parent 2020-04-10T07:09:46.303 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1 2020-04-10T07:09:46.303 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:46.303 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror_parent 2020-04-10T07:09:46.303 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_parent 2020-04-10T07:09:46.304 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster2-mirror_parent-test_parent.status 2020-04-10T07:09:46.309 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2-mirror_parent-test_parent.status 2020-04-10T07:09:46.309 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:09:46.309 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster2-mirror_parent-test_parent.status 2020-04-10T07:09:46.311 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror_parent-test_parent.status 2020-04-10T07:09:46.311 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror_parent-test_parent.status 2020-04-10T07:09:46.311 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:09:46.311 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror_parent-test_parent.status 2020-04-10T07:09:46.311 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mirror_image_snapshot cluster2 mirror_parent test_parent 2020-04-10T07:09:46.311 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:46.312 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror_parent 2020-04-10T07:09:46.312 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_parent 2020-04-10T07:09:46.312 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror_parent/test_parent 2020-04-10T07:09:47.171 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Snapshot ID: 6 2020-04-10T07:09:47.180 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_newest_mirror_snapshot cluster2 mirror_parent test_parent /tmp/tmp.8SwBCsYPSC/cluster2-mirror_parent-test_parent.status 2020-04-10T07:09:47.181 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:47.181 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror_parent 2020-04-10T07:09:47.181 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_parent 2020-04-10T07:09:47.181 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local log=/tmp/tmp.8SwBCsYPSC/cluster2-mirror_parent-test_parent.status 2020-04-10T07:09:47.181 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 snap list --all mirror_parent/test_parent --format xml 2020-04-10T07:09:47.181 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2020-04-10T07:09:47.226 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //snapshot/id 2020-04-10T07:09:47.229 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snapshot_id=6 2020-04-10T07:09:47.229 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ true 2020-04-10T07:09:47.229 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0.2 2020-04-10T07:09:47.430 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_newest_mirror_snapshot cluster1 mirror_parent test_parent /tmp/tmp.8SwBCsYPSC/cluster1-mirror_parent-test_parent.status 2020-04-10T07:09:47.430 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:47.430 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror_parent 2020-04-10T07:09:47.430 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_parent 2020-04-10T07:09:47.430 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror_parent-test_parent.status 2020-04-10T07:09:47.431 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 snap list --all mirror_parent/test_parent --format xml 2020-04-10T07:09:47.431 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2020-04-10T07:09:47.494 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2020-04-10T07:09:47.500 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local primary_snapshot_id=4 2020-04-10T07:09:47.500 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test 6 = 4 2020-04-10T07:09:47.500 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0.4 2020-04-10T07:09:47.902 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_newest_mirror_snapshot cluster1 mirror_parent test_parent /tmp/tmp.8SwBCsYPSC/cluster1-mirror_parent-test_parent.status 2020-04-10T07:09:47.902 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:47.902 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror_parent 2020-04-10T07:09:47.903 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_parent 2020-04-10T07:09:47.903 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror_parent-test_parent.status 2020-04-10T07:09:47.903 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 snap list --all mirror_parent/test_parent --format xml 2020-04-10T07:09:47.903 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2020-04-10T07:09:47.967 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2020-04-10T07:09:47.973 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local primary_snapshot_id=4 2020-04-10T07:09:47.973 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test 6 = 4 2020-04-10T07:09:47.974 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0.8 2020-04-10T07:09:48.774 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_newest_mirror_snapshot cluster1 mirror_parent test_parent /tmp/tmp.8SwBCsYPSC/cluster1-mirror_parent-test_parent.status 2020-04-10T07:09:48.775 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:48.775 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror_parent 2020-04-10T07:09:48.775 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_parent 2020-04-10T07:09:48.775 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror_parent-test_parent.status 2020-04-10T07:09:48.775 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 snap list --all mirror_parent/test_parent --format xml 2020-04-10T07:09:48.775 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2020-04-10T07:09:48.841 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ true 2020-04-10T07:09:48.841 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2020-04-10T07:09:48.844 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:-:1.1: Document is empty 2020-04-10T07:09:48.844 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: 2020-04-10T07:09:48.844 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:^ 2020-04-10T07:09:48.845 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local primary_snapshot_id= 2020-04-10T07:09:48.845 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test 6 = 2020-04-10T07:09:48.845 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1.6 2020-04-10T07:09:50.450 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_newest_mirror_snapshot cluster1 mirror_parent test_parent /tmp/tmp.8SwBCsYPSC/cluster1-mirror_parent-test_parent.status 2020-04-10T07:09:50.450 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:50.450 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror_parent 2020-04-10T07:09:50.451 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_parent 2020-04-10T07:09:50.451 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror_parent-test_parent.status 2020-04-10T07:09:50.451 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 snap list --all mirror_parent/test_parent --format xml 2020-04-10T07:09:50.451 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2020-04-10T07:09:50.513 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2020-04-10T07:09:50.518 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local primary_snapshot_id=6 2020-04-10T07:09:50.518 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test 6 = 6 2020-04-10T07:09:50.519 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:09:50.519 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_status_in_pool_dir cluster1 mirror_parent test_parent up+replaying 2020-04-10T07:09:50.519 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:50.519 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror_parent 2020-04-10T07:09:50.519 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_parent 2020-04-10T07:09:50.519 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:09:50.519 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:09:50.520 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:09:50.520 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:09:51.520 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_status_in_pool_dir cluster1 mirror_parent test_parent up+replaying 2020-04-10T07:09:51.521 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:51.521 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror_parent 2020-04-10T07:09:51.521 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_parent 2020-04-10T07:09:51.521 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:09:51.521 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:09:51.521 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:09:51.522 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror_parent-test_parent.mirror_status 2020-04-10T07:09:51.522 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror_parent-test_parent.mirror_status 2020-04-10T07:09:51.522 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:09:51.524 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror_parent-test_parent.mirror_status 2020-04-10T07:09:51.524 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror_parent/test_parent 2020-04-10T07:09:51.524 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ tee /tmp/tmp.8SwBCsYPSC/cluster1-mirror_parent-test_parent.mirror_status 2020-04-10T07:09:51.594 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:test_parent: 2020-04-10T07:09:51.595 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: global_id: dd523e97-79dd-49a7-b2c9-df255e9660d4 2020-04-10T07:09:51.595 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+replaying 2020-04-10T07:09:51.595 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: replaying, {"replay_state": "idle", "remote_snapshot_timestamp": 1586502583} 2020-04-10T07:09:51.595 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: service: mirror.0 on smithi204 2020-04-10T07:09:51.595 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:09:45 2020-04-10T07:09:51.595 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: peer_sites: 2020-04-10T07:09:51.596 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: name: cluster2-DNE 2020-04-10T07:09:51.596 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+stopped 2020-04-10T07:09:51.596 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: local image is primary 2020-04-10T07:09:51.596 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:09:51 2020-04-10T07:09:51.604 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ state: .*up+replaying /tmp/tmp.8SwBCsYPSC/cluster1-mirror_parent-test_parent.mirror_status 2020-04-10T07:09:51.605 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+replaying 2020-04-10T07:09:51.605 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ description: .* /tmp/tmp.8SwBCsYPSC/cluster1-mirror_parent-test_parent.mirror_status 2020-04-10T07:09:51.615 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: replaying, {"replay_state": "idle", "remote_snapshot_timestamp": 1586502583} 2020-04-10T07:09:51.615 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+replaying 2020-04-10T07:09:51.615 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: service: mirror.0 on smithi204 2020-04-10T07:09:51.615 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:09:51.616 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+replaying 2020-04-10T07:09:51.616 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^up+ 2020-04-10T07:09:51.616 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep service: *mirror..* on /tmp/tmp.8SwBCsYPSC/cluster1-mirror_parent-test_parent.mirror_status 2020-04-10T07:09:51.618 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.8SwBCsYPSC/cluster1-mirror_parent-test_parent.mirror_status 2020-04-10T07:09:51.618 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update=2020-04-10 07:09:45 2020-04-10T07:09:51.618 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_mirror_pool_status_verbose cluster1 mirror_parent test_parent up+replaying 2020-04-10 07:09:45 2020-04-10T07:09:51.618 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:51.618 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror_parent 2020-04-10T07:09:51.618 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_parent 2020-04-10T07:09:51.618 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:09:51.619 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local prev_last_update=2020-04-10 07:09:45 2020-04-10T07:09:51.619 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror_parent.mirror_status 2020-04-10T07:09:51.619 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror_parent.mirror_status 2020-04-10T07:09:51.619 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:09:51.619 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror_parent.mirror_status 2020-04-10T07:09:51.619 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 mirror pool status mirror_parent --verbose --format xml 2020-04-10T07:09:51.661 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: failed to query services: (13) Permission denied 2020-04-10T07:09:51.710 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update state 2020-04-10T07:09:51.711 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test_parent']/last_update 2020-04-10T07:09:51.713 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ last_update=2020-04-10 07:09:45 2020-04-10T07:09:51.714 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test_parent']/state 2020-04-10T07:09:51.720 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ state=up+replaying 2020-04-10T07:09:51.720 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+replaying 2020-04-10T07:09:51.720 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep up+replaying 2020-04-10T07:09:51.721 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+replaying 2020-04-10T07:09:51.722 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:09:51.722 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:09:51.722 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ compare_images mirror_parent test_parent 2020-04-10T07:09:51.723 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror_parent 2020-04-10T07:09:51.723 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_parent 2020-04-10T07:09:51.723 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local ret=0 2020-04-10T07:09:51.723 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster2-mirror_parent-test_parent.export 2020-04-10T07:09:51.723 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2-mirror_parent-test_parent.export 2020-04-10T07:09:51.723 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:09:51.724 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local rmt_export=/tmp/tmp.8SwBCsYPSC/cluster2-mirror_parent-test_parent.export 2020-04-10T07:09:51.724 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror_parent-test_parent.export 2020-04-10T07:09:51.725 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror_parent-test_parent.export 2020-04-10T07:09:51.725 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:09:51.725 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local loc_export=/tmp/tmp.8SwBCsYPSC/cluster1-mirror_parent-test_parent.export 2020-04-10T07:09:51.726 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rm -f /tmp/tmp.8SwBCsYPSC/cluster2-mirror_parent-test_parent.export /tmp/tmp.8SwBCsYPSC/cluster1-mirror_parent-test_parent.export 2020-04-10T07:09:51.727 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 export mirror_parent/test_parent /tmp/tmp.8SwBCsYPSC/cluster2-mirror_parent-test_parent.export 2020-04-10T07:09:51.894 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2020-04-10T07:09:51.897 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 export mirror_parent/test_parent /tmp/tmp.8SwBCsYPSC/cluster1-mirror_parent-test_parent.export 2020-04-10T07:09:52.064 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2020-04-10T07:09:52.068 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cmp /tmp/tmp.8SwBCsYPSC/cluster2-mirror_parent-test_parent.export /tmp/tmp.8SwBCsYPSC/cluster1-mirror_parent-test_parent.export 2020-04-10T07:09:52.142 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rm -f /tmp/tmp.8SwBCsYPSC/cluster2-mirror_parent-test_parent.export /tmp/tmp.8SwBCsYPSC/cluster1-mirror_parent-test_parent.export 2020-04-10T07:09:52.174 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:09:52.174 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_started cluster1 mirror test_clone 2020-04-10T07:09:52.174 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:52.174 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:52.175 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_clone 2020-04-10T07:09:52.175 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_state cluster1 mirror test_clone started 2020-04-10T07:09:52.175 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:52.175 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:52.175 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_clone 2020-04-10T07:09:52.175 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state=started 2020-04-10T07:09:52.175 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local s 2020-04-10T07:09:52.176 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:09:53.174 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_image_replay_state cluster1 mirror test_clone started 2020-04-10T07:09:53.174 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:53.174 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:53.174 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_clone 2020-04-10T07:09:53.174 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local test_state=started 2020-04-10T07:09:53.175 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_result 2020-04-10T07:09:53.175 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_state=stopped 2020-04-10T07:09:53.176 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test_clone 2020-04-10T07:09:53.176 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_instance=cluster1 2020-04-10T07:09:53.176 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:09:53.176 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:53.177 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ + local instance=cluster1 2020-04-10T07:09:53.177 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local loop_instance 2020-04-10T07:09:53.177 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:grep -i state 2020-04-10T07:09:53.177 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0 2020-04-10T07:09:53.177 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ cluster1 != cluster1 ] 2020-04-10T07:09:53.177 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:09:53.180 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test_clone 2020-04-10T07:09:53.180 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:09:53.180 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:09:53.180 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:09:53.180 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:09:53.180 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:09:53.189 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:09:53.189 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:09:53.189 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:09:53.189 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:09:53.189 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:09:53.189 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:09:53.189 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:09:53.190 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:09:53.190 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:09:53.190 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster1 2020-04-10T07:09:53.190 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:09:53.190 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:53.190 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:09:53.190 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:09:53.191 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:09:53.191 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:09:53.191 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:09:53.191 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:09:53.191 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:09:53.191 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:09:53.191 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:09:53.192 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:09:53.192 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:09:53.192 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:09:53.192 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:09:53.201 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:09:53.202 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:09:53.202 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:09:53.202 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test_clone 2020-04-10T07:09:53.306 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:09:53.307 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ status_result= "state": "Replaying" 2020-04-10T07:09:53.307 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo "state": "Replaying" 2020-04-10T07:09:53.307 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep -i Replaying 2020-04-10T07:09:53.308 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "state": "Replaying" 2020-04-10T07:09:53.314 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ current_state=started 2020-04-10T07:09:53.314 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test started = started 2020-04-10T07:09:53.314 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:09:53.314 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror test_clone 2020-04-10T07:09:53.314 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1 2020-04-10T07:09:53.315 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:53.315 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:53.315 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_clone 2020-04-10T07:09:53.315 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ snapshot = journal ] 2020-04-10T07:09:53.315 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ snapshot = snapshot ] 2020-04-10T07:09:53.315 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror test_clone 2020-04-10T07:09:53.315 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1 2020-04-10T07:09:53.316 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:53.316 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:53.316 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_clone 2020-04-10T07:09:53.316 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster2-mirror-test_clone.status 2020-04-10T07:09:53.333 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2-mirror-test_clone.status 2020-04-10T07:09:53.333 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:09:53.333 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster2-mirror-test_clone.status 2020-04-10T07:09:53.333 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror-test_clone.status 2020-04-10T07:09:53.333 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ + echo cluster1-mirror-test_clone.status 2020-04-10T07:09:53.333 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:sed -e s|[/ ]|_|g 2020-04-10T07:09:53.334 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_clone.status 2020-04-10T07:09:53.334 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mirror_image_snapshot cluster2 mirror test_clone 2020-04-10T07:09:53.334 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:53.334 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:53.334 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_clone 2020-04-10T07:09:53.334 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test_clone 2020-04-10T07:09:54.124 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Snapshot ID: 40 2020-04-10T07:09:54.136 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_newest_mirror_snapshot cluster2 mirror test_clone /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test_clone.status 2020-04-10T07:09:54.136 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:54.136 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:54.137 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_clone 2020-04-10T07:09:54.137 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local log=/tmp/tmp.8SwBCsYPSC/cluster2-mirror-test_clone.status 2020-04-10T07:09:54.137 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 snap list --all mirror/test_clone --format xml 2020-04-10T07:09:54.137 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2020-04-10T07:09:54.204 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //snapshot/id 2020-04-10T07:09:54.207 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snapshot_id=40 2020-04-10T07:09:54.208 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ true 2020-04-10T07:09:54.208 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0.2 2020-04-10T07:09:54.409 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_newest_mirror_snapshot cluster1 mirror test_clone /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_clone.status 2020-04-10T07:09:54.410 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:54.410 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:54.410 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_clone 2020-04-10T07:09:54.410 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_clone.status 2020-04-10T07:09:54.410 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 snap list --all mirror/test_clone --format xml 2020-04-10T07:09:54.410 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2020-04-10T07:09:54.490 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2020-04-10T07:09:54.496 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local primary_snapshot_id=39 2020-04-10T07:09:54.496 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test 40 = 39 2020-04-10T07:09:54.496 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0.4 2020-04-10T07:09:54.897 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_newest_mirror_snapshot cluster1 mirror test_clone /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_clone.status 2020-04-10T07:09:54.897 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:54.898 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:54.898 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_clone 2020-04-10T07:09:54.898 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_clone.status 2020-04-10T07:09:54.898 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 snap list --all mirror/test_clone --format xml 2020-04-10T07:09:54.898 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2020-04-10T07:09:54.990 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2020-04-10T07:09:54.990 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local primary_snapshot_id=39 2020-04-10T07:09:54.990 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test 40 = 39 2020-04-10T07:09:54.990 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0.8 2020-04-10T07:09:55.786 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_newest_mirror_snapshot cluster1 mirror test_clone /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_clone.status 2020-04-10T07:09:55.787 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:55.787 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:55.787 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_clone 2020-04-10T07:09:55.787 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_clone.status 2020-04-10T07:09:55.787 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 snap list --all mirror/test_clone --format xml 2020-04-10T07:09:55.787 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2020-04-10T07:09:55.880 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2020-04-10T07:09:55.885 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local primary_snapshot_id=40 2020-04-10T07:09:55.886 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test 40 = 40 2020-04-10T07:09:55.886 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:09:55.886 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test_clone up+replaying 2020-04-10T07:09:55.886 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:55.886 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:55.886 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_clone 2020-04-10T07:09:55.887 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:09:55.887 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:09:55.887 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:09:55.887 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:09:56.889 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_status_in_pool_dir cluster1 mirror test_clone up+replaying 2020-04-10T07:09:56.889 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:56.889 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:56.890 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_clone 2020-04-10T07:09:56.890 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:09:56.890 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:09:56.890 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:09:56.890 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror-test_clone.mirror_status 2020-04-10T07:09:56.890 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror-test_clone.mirror_status 2020-04-10T07:09:56.890 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:09:56.891 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_clone.mirror_status 2020-04-10T07:09:56.891 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test_clone 2020-04-10T07:09:56.891 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ tee /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_clone.mirror_status 2020-04-10T07:09:56.984 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:test_clone: 2020-04-10T07:09:56.984 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: global_id: 003078e2-ed6c-4f36-8bc5-ce0c44ef25d5 2020-04-10T07:09:56.984 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+replaying 2020-04-10T07:09:56.984 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: replaying, {"replay_state": "idle", "remote_snapshot_timestamp": 1586502585} 2020-04-10T07:09:56.985 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: service: mirror.0 on smithi204 2020-04-10T07:09:56.985 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:09:49 2020-04-10T07:09:56.985 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: peer_sites: 2020-04-10T07:09:56.985 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: name: cluster2-DNE 2020-04-10T07:09:56.985 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+stopped 2020-04-10T07:09:56.985 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: local image is primary 2020-04-10T07:09:56.985 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:09:56 2020-04-10T07:09:56.993 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ state: .*up+replaying /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_clone.mirror_status 2020-04-10T07:09:56.995 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+replaying 2020-04-10T07:09:56.996 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ description: .* /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_clone.mirror_status 2020-04-10T07:09:56.997 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: replaying, {"replay_state": "idle", "remote_snapshot_timestamp": 1586502585} 2020-04-10T07:09:56.998 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:09:56.998 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ + echo up+replaying 2020-04-10T07:09:56.998 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:grep ^up+ 2020-04-10T07:09:56.999 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+replaying 2020-04-10T07:09:56.999 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep service: *mirror..* on /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_clone.mirror_status 2020-04-10T07:09:57.005 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: service: mirror.0 on smithi204 2020-04-10T07:09:57.005 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_clone.mirror_status 2020-04-10T07:09:57.005 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update=2020-04-10 07:09:49 2020-04-10T07:09:57.005 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test_clone up+replaying 2020-04-10 07:09:49 2020-04-10T07:09:57.005 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:57.006 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:57.006 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_clone 2020-04-10T07:09:57.006 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:09:57.006 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local prev_last_update=2020-04-10 07:09:49 2020-04-10T07:09:57.006 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror.mirror_status 2020-04-10T07:09:57.006 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror.mirror_status 2020-04-10T07:09:57.006 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:09:57.007 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror.mirror_status 2020-04-10T07:09:57.007 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2020-04-10T07:09:57.051 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: failed to query services: (13) Permission denied 2020-04-10T07:09:57.121 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update state 2020-04-10T07:09:57.121 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test_clone']/last_update 2020-04-10T07:09:57.127 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ last_update=2020-04-10 07:09:49 2020-04-10T07:09:57.127 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test_clone']/state 2020-04-10T07:09:57.131 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ state=up+replaying 2020-04-10T07:09:57.132 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+replaying 2020-04-10T07:09:57.132 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep up+replaying 2020-04-10T07:09:57.135 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+replaying 2020-04-10T07:09:57.135 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:09:57.135 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:09:57.135 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ compare_images mirror test_clone 2020-04-10T07:09:57.135 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:57.135 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_clone 2020-04-10T07:09:57.135 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local ret=0 2020-04-10T07:09:57.136 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster2-mirror-test_clone.export 2020-04-10T07:09:57.136 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2-mirror-test_clone.export 2020-04-10T07:09:57.136 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:09:57.138 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local rmt_export=/tmp/tmp.8SwBCsYPSC/cluster2-mirror-test_clone.export 2020-04-10T07:09:57.138 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror-test_clone.export 2020-04-10T07:09:57.138 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror-test_clone.export 2020-04-10T07:09:57.138 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:09:57.139 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local loc_export=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_clone.export 2020-04-10T07:09:57.139 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rm -f /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test_clone.export /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_clone.export 2020-04-10T07:09:57.139 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 export mirror/test_clone /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test_clone.export 2020-04-10T07:09:57.345 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2020-04-10T07:09:57.349 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 export mirror/test_clone /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_clone.export 2020-04-10T07:09:57.570 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2020-04-10T07:09:57.573 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cmp /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test_clone.export /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_clone.export 2020-04-10T07:09:57.645 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rm -f /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test_clone.export /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_clone.export 2020-04-10T07:09:57.682 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:09:57.683 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ testlog - clone v1 2020-04-10T07:09:57.683 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ tee -a /tmp/tmp.8SwBCsYPSC/rbd-mirror.test.log 2020-04-10T07:09:57.683 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ date +%F %T 2020-04-10T07:09:57.684 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10 07:09:57 - clone v1 2020-04-10T07:09:57.684 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:2020-04-10 07:09:57 - clone v1 2020-04-10T07:09:57.684 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ clone_image_and_enable_mirror cluster1 mirror_parent test_parent snap mirror test_clone1 2020-04-10T07:09:57.684 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:57.688 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local parent_pool=mirror_parent 2020-04-10T07:09:57.688 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local parent_image=test_parent 2020-04-10T07:09:57.688 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local parent_snap=snap 2020-04-10T07:09:57.688 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local clone_pool=mirror 2020-04-10T07:09:57.689 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local clone_image=test_clone1 2020-04-10T07:09:57.689 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 6 2020-04-10T07:09:57.689 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local mode=snapshot 2020-04-10T07:09:57.689 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:09:57.689 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ clone_image cluster1 mirror_parent test_parent snap mirror test_clone1 2020-04-10T07:09:57.689 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:57.689 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local parent_pool=mirror_parent 2020-04-10T07:09:57.690 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local parent_image=test_parent 2020-04-10T07:09:57.690 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local parent_snap=snap 2020-04-10T07:09:57.691 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local clone_pool=mirror 2020-04-10T07:09:57.692 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local clone_image=test_clone1 2020-04-10T07:09:57.692 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 6 2020-04-10T07:09:57.692 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 clone mirror_parent/test_parent@snap mirror/test_clone1 --image-feature layering,exclusive-lock,journaling 2020-04-10T07:09:57.823 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ enable_mirror cluster1 mirror test_clone1 snapshot 2020-04-10T07:09:57.823 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:57.823 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:57.823 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_clone1 2020-04-10T07:09:57.824 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local mode=snapshot 2020-04-10T07:09:57.824 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/test_clone1 snapshot 2020-04-10T07:09:58.137 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Mirroring enabled 2020-04-10T07:09:58.152 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ clone_image_and_enable_mirror cluster2 mirror_parent test_parent snap mirror test_clone_v1 snapshot --rbd-default-clone-format 1 2020-04-10T07:09:58.152 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:58.152 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local parent_pool=mirror_parent 2020-04-10T07:09:58.152 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local parent_image=test_parent 2020-04-10T07:09:58.153 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local parent_snap=snap 2020-04-10T07:09:58.153 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local clone_pool=mirror 2020-04-10T07:09:58.153 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local clone_image=test_clone_v1 2020-04-10T07:09:58.153 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 6 2020-04-10T07:09:58.153 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local mode=snapshot 2020-04-10T07:09:58.153 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n snapshot ] 2020-04-10T07:09:58.153 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:09:58.154 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ clone_image cluster2 mirror_parent test_parent snap mirror test_clone_v1 --rbd-default-clone-format 1 2020-04-10T07:09:58.154 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:58.154 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local parent_pool=mirror_parent 2020-04-10T07:09:58.154 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local parent_image=test_parent 2020-04-10T07:09:58.154 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local parent_snap=snap 2020-04-10T07:09:58.154 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local clone_pool=mirror 2020-04-10T07:09:58.154 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local clone_image=test_clone_v1 2020-04-10T07:09:58.155 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 6 2020-04-10T07:09:58.155 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 clone mirror_parent/test_parent@snap mirror/test_clone_v1 --image-feature layering,exclusive-lock,journaling --rbd-default-clone-format 1 2020-04-10T07:09:58.264 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ enable_mirror cluster2 mirror test_clone_v1 snapshot 2020-04-10T07:09:58.264 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:58.264 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:58.265 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_clone_v1 2020-04-10T07:09:58.265 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local mode=snapshot 2020-04-10T07:09:58.265 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/test_clone_v1 snapshot 2020-04-10T07:09:59.125 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Mirroring enabled 2020-04-10T07:09:59.135 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_clone_format cluster2 mirror test_clone_v1 2020-04-10T07:09:59.135 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:09:59.135 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:59.135 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_clone_v1 2020-04-10T07:09:59.135 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ awk BEGIN { 2020-04-10T07:09:59.136 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: format = 1 2020-04-10T07:09:59.136 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: } 2020-04-10T07:09:59.136 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: $1 == "parent:" { 2020-04-10T07:09:59.136 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: parent = $2 2020-04-10T07:09:59.136 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: } 2020-04-10T07:09:59.136 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: /op_features: .*clone-child/ { 2020-04-10T07:09:59.136 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: format = 2 2020-04-10T07:09:59.137 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: } 2020-04-10T07:09:59.137 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: END { 2020-04-10T07:09:59.137 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: if (!parent) exit 1 2020-04-10T07:09:59.137 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: print format 2020-04-10T07:09:59.137 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: } 2020-04-10T07:09:59.137 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 info mirror/test_clone_v1 2020-04-10T07:09:59.195 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test 1 = 1 2020-04-10T07:09:59.195 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_started cluster1 mirror test_clone_v1 2020-04-10T07:09:59.196 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:59.196 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:59.196 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_clone_v1 2020-04-10T07:09:59.196 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_state cluster1 mirror test_clone_v1 started 2020-04-10T07:09:59.196 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:09:59.196 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:09:59.196 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_clone_v1 2020-04-10T07:09:59.197 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state=started 2020-04-10T07:09:59.197 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local s 2020-04-10T07:09:59.197 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:10:00.200 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_image_replay_state cluster1 mirror test_clone_v1 started 2020-04-10T07:10:00.323 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:00.323 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:00.323 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_clone_v1 2020-04-10T07:10:00.323 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local test_state=started 2020-04-10T07:10:00.323 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_result 2020-04-10T07:10:00.324 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_state=stopped 2020-04-10T07:10:00.324 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test_clone_v1 2020-04-10T07:10:00.324 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_instance=cluster1 2020-04-10T07:10:00.324 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:10:00.324 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:00.324 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance=cluster1 2020-04-10T07:10:00.325 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local loop_instance 2020-04-10T07:10:00.325 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0 2020-04-10T07:10:00.325 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep -i state 2020-04-10T07:10:00.325 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ cluster1 != cluster1 ] 2020-04-10T07:10:00.325 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:10:00.325 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test_clone_v1 2020-04-10T07:10:00.325 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:10:00.325 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:10:00.326 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:10:00.326 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:10:00.326 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:10:00.326 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:10:00.326 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:10:00.326 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:10:00.326 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:10:00.327 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:10:00.327 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:10:00.327 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:10:00.327 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:10:00.327 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:10:00.327 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster1 2020-04-10T07:10:00.327 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:10:00.327 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:00.328 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:10:00.328 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:10:00.328 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:10:00.328 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:10:00.328 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:10:00.328 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:10:00.330 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:10:00.330 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:10:00.330 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:10:00.330 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:10:00.330 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:10:00.330 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:10:00.331 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:10:00.335 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:10:00.335 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:10:00.335 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:10:00.335 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test_clone_v1 2020-04-10T07:10:00.335 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:Invalid command: unused arguments: ['mirror/test_clone_v1'] 2020-04-10T07:10:00.335 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status : get status for rbd mirror 2020-04-10T07:10:00.335 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:10:00.336 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test_clone_v1 2020-04-10T07:10:00.336 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:1 2020-04-10T07:10:00.336 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:10:00.336 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:10:00.336 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 cluster instance 2020-04-10T07:10:00.336 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:10:00.336 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:10:00.337 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:10:00.337 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:10:00.337 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:10:00.337 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:10:00.337 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:10:00.337 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:10:00.337 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:10:00.337 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:10:00.338 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:1 cluster1 2020-04-10T07:10:00.338 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:1 2020-04-10T07:10:00.338 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:00.339 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:10:00.339 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2020-04-10T07:10:00.339 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:10:00.340 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:10:00.340 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:10:00.340 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:10:00.340 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:10:00.340 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:10:00.340 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:10:00.340 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:10:00.341 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:10:00.341 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:10:00.341 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2020-04-10T07:10:00.341 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:10:00.341 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:10:00.341 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:10:00.341 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test_clone_v1 2020-04-10T07:10:00.424 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:Invalid command: unused arguments: ['mirror/test_clone_v1'] 2020-04-10T07:10:00.424 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status : get status for rbd mirror 2020-04-10T07:10:00.424 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:10:00.428 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:2 rbd mirror status mirror/test_clone_v1 2020-04-10T07:10:00.428 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:2 2020-04-10T07:10:00.428 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:10:00.428 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:10:00.428 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 cluster instance 2020-04-10T07:10:00.428 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:10:00.428 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:10:00.429 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:10:00.429 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:10:00.429 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:10:00.429 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:10:00.429 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:10:00.429 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:10:00.430 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:10:00.430 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:10:00.430 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:2 cluster1 2020-04-10T07:10:00.430 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:2 2020-04-10T07:10:00.430 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:00.430 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:10:00.430 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2020-04-10T07:10:00.431 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:10:00.431 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:10:00.431 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:10:00.431 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:10:00.431 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:10:00.431 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:10:00.431 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:10:00.431 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:10:00.432 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:10:00.432 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:10:00.432 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2020-04-10T07:10:00.449 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:10:00.450 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:10:00.452 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:10:00.452 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test_clone_v1 2020-04-10T07:10:00.544 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:10:00.545 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ status_result= "state": "Replaying" 2020-04-10T07:10:00.545 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo "state": "Replaying" 2020-04-10T07:10:00.545 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep -i Replaying 2020-04-10T07:10:00.547 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "state": "Replaying" 2020-04-10T07:10:00.548 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ current_state=started 2020-04-10T07:10:00.548 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test started = started 2020-04-10T07:10:00.549 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:10:00.549 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_clone_format cluster1 mirror test_clone_v1 2020-04-10T07:10:00.549 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:00.549 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:00.549 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_clone_v1 2020-04-10T07:10:00.550 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 info mirror/test_clone_v1 2020-04-10T07:10:00.550 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ awk BEGIN { 2020-04-10T07:10:00.550 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: format = 1 2020-04-10T07:10:00.550 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: } 2020-04-10T07:10:00.551 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: $1 == "parent:" { 2020-04-10T07:10:00.551 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: parent = $2 2020-04-10T07:10:00.551 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: } 2020-04-10T07:10:00.551 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: /op_features: .*clone-child/ { 2020-04-10T07:10:00.552 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: format = 2 2020-04-10T07:10:00.552 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: } 2020-04-10T07:10:00.553 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: END { 2020-04-10T07:10:00.553 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: if (!parent) exit 1 2020-04-10T07:10:00.553 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: print format 2020-04-10T07:10:00.553 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: } 2020-04-10T07:10:00.630 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test 1 = 1 2020-04-10T07:10:00.631 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ testlog - clone v2 2020-04-10T07:10:00.631 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ tee -a /tmp/tmp.8SwBCsYPSC/rbd-mirror.test.log 2020-04-10T07:10:00.631 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ date +%F %T 2020-04-10T07:10:00.633 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10 07:10:00 - clone v2 2020-04-10T07:10:00.633 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:2020-04-10 07:10:00 - clone v2 2020-04-10T07:10:00.633 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ parent_snap=snap_v2 2020-04-10T07:10:00.633 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ create_snapshot cluster2 mirror_parent test_parent snap_v2 2020-04-10T07:10:00.634 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:00.634 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror_parent 2020-04-10T07:10:00.634 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_parent 2020-04-10T07:10:00.634 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snap=snap_v2 2020-04-10T07:10:00.634 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 snap create mirror_parent/test_parent@snap_v2 2020-04-10T07:10:01.128 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mirror_image_snapshot cluster2 mirror_parent test_parent 2020-04-10T07:10:01.128 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:01.128 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror_parent 2020-04-10T07:10:01.129 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_parent 2020-04-10T07:10:01.129 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror_parent/test_parent 2020-04-10T07:10:02.158 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Snapshot ID: 9 2020-04-10T07:10:02.168 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ clone_image_and_enable_mirror cluster2 mirror_parent test_parent snap_v2 mirror test_clone_v2 snapshot --rbd-default-clone-format 2 2020-04-10T07:10:02.168 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:02.168 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local parent_pool=mirror_parent 2020-04-10T07:10:02.168 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local parent_image=test_parent 2020-04-10T07:10:02.168 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local parent_snap=snap_v2 2020-04-10T07:10:02.169 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local clone_pool=mirror 2020-04-10T07:10:02.169 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local clone_image=test_clone_v2 2020-04-10T07:10:02.169 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 6 2020-04-10T07:10:02.169 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local mode=snapshot 2020-04-10T07:10:02.169 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n snapshot ] 2020-04-10T07:10:02.170 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:10:02.170 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ clone_image cluster2 mirror_parent test_parent snap_v2 mirror test_clone_v2 --rbd-default-clone-format 2 2020-04-10T07:10:02.170 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:02.170 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local parent_pool=mirror_parent 2020-04-10T07:10:02.170 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local parent_image=test_parent 2020-04-10T07:10:02.171 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local parent_snap=snap_v2 2020-04-10T07:10:02.171 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local clone_pool=mirror 2020-04-10T07:10:02.171 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local clone_image=test_clone_v2 2020-04-10T07:10:02.171 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 6 2020-04-10T07:10:02.171 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 clone mirror_parent/test_parent@snap_v2 mirror/test_clone_v2 --image-feature layering,exclusive-lock,journaling --rbd-default-clone-format 2 2020-04-10T07:10:02.258 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ enable_mirror cluster2 mirror test_clone_v2 snapshot 2020-04-10T07:10:02.258 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:02.259 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:02.259 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_clone_v2 2020-04-10T07:10:02.259 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local mode=snapshot 2020-04-10T07:10:02.259 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/test_clone_v2 snapshot 2020-04-10T07:10:03.172 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Mirroring enabled 2020-04-10T07:10:03.181 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_clone_format cluster2 mirror test_clone_v2 2020-04-10T07:10:03.181 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:03.182 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:03.182 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_clone_v2 2020-04-10T07:10:03.182 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 info mirror/test_clone_v2 2020-04-10T07:10:03.182 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ awk BEGIN { 2020-04-10T07:10:03.182 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: format = 1 2020-04-10T07:10:03.183 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: } 2020-04-10T07:10:03.183 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: $1 == "parent:" { 2020-04-10T07:10:03.183 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: parent = $2 2020-04-10T07:10:03.183 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: } 2020-04-10T07:10:03.183 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: /op_features: .*clone-child/ { 2020-04-10T07:10:03.183 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: format = 2 2020-04-10T07:10:03.183 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: } 2020-04-10T07:10:03.184 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: END { 2020-04-10T07:10:03.184 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: if (!parent) exit 1 2020-04-10T07:10:03.184 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: print format 2020-04-10T07:10:03.184 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: } 2020-04-10T07:10:03.239 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test 2 = 2 2020-04-10T07:10:03.239 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_started cluster1 mirror test_clone_v2 2020-04-10T07:10:03.239 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:03.239 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:03.240 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_clone_v2 2020-04-10T07:10:03.240 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_state cluster1 mirror test_clone_v2 started 2020-04-10T07:10:03.240 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:03.240 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:03.248 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_clone_v2 2020-04-10T07:10:03.249 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state=started 2020-04-10T07:10:03.249 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local s 2020-04-10T07:10:03.249 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:10:04.239 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_image_replay_state cluster1 mirror test_clone_v2 started 2020-04-10T07:10:04.239 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:04.239 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:04.239 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_clone_v2 2020-04-10T07:10:04.239 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local test_state=started 2020-04-10T07:10:04.240 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_result 2020-04-10T07:10:04.240 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_state=stopped 2020-04-10T07:10:04.241 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test_clone_v2 2020-04-10T07:10:04.241 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ + local cluster_instance=cluster1 2020-04-10T07:10:04.241 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:grep+ -ishift state 2020-04-10T07:10:04.241 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: 2020-04-10T07:10:04.242 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:04.242 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance=cluster1 2020-04-10T07:10:04.242 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local loop_instance 2020-04-10T07:10:04.242 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0 2020-04-10T07:10:04.242 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ cluster1 != cluster1 ] 2020-04-10T07:10:04.242 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:10:04.245 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test_clone_v2 2020-04-10T07:10:04.245 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:10:04.245 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:10:04.245 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:10:04.245 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:10:04.246 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:10:04.246 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:10:04.246 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:10:04.246 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:10:04.246 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:10:04.246 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:10:04.246 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:10:04.247 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:10:04.247 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:10:04.247 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:10:04.247 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster1 2020-04-10T07:10:04.247 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:10:04.247 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:04.247 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:10:04.248 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:10:04.248 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:10:04.248 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:10:04.248 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:10:04.248 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:10:04.248 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:10:04.248 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:10:04.249 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:10:04.249 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:10:04.249 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:10:04.249 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:10:04.249 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:10:04.265 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:10:04.267 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:10:04.267 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:10:04.267 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test_clone_v2 2020-04-10T07:10:04.362 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:Invalid command: unused arguments: ['mirror/test_clone_v2'] 2020-04-10T07:10:04.362 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status : get status for rbd mirror 2020-04-10T07:10:04.362 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:10:04.364 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test_clone_v2 2020-04-10T07:10:04.364 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:1 2020-04-10T07:10:04.365 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:10:04.365 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:10:04.365 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 cluster instance 2020-04-10T07:10:04.365 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:10:04.365 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:10:04.365 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:10:04.365 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:10:04.366 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:10:04.367 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:10:04.367 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:10:04.368 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:10:04.368 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:10:04.368 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:10:04.368 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:1 cluster1 2020-04-10T07:10:04.368 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:1 2020-04-10T07:10:04.368 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:04.368 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:10:04.369 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2020-04-10T07:10:04.369 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:10:04.369 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:10:04.369 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:10:04.369 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:10:04.369 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:10:04.369 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:10:04.370 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:10:04.370 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:10:04.370 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:10:04.370 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:10:04.370 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2020-04-10T07:10:04.386 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:10:04.386 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:10:04.386 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:10:04.386 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test_clone_v2 2020-04-10T07:10:04.471 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:Invalid command: unused arguments: ['mirror/test_clone_v2'] 2020-04-10T07:10:04.471 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status : get status for rbd mirror 2020-04-10T07:10:04.471 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:10:04.476 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:2 rbd mirror status mirror/test_clone_v2 2020-04-10T07:10:04.477 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:2 2020-04-10T07:10:04.477 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:10:04.477 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:10:04.477 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 cluster instance 2020-04-10T07:10:04.477 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:10:04.477 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:10:04.478 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:10:04.478 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:10:04.478 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:10:04.478 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:10:04.478 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:10:04.478 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:10:04.478 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:10:04.479 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:10:04.479 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:2 cluster1 2020-04-10T07:10:04.479 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:2 2020-04-10T07:10:04.479 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:04.479 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:10:04.479 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2020-04-10T07:10:04.479 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:10:04.480 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:10:04.480 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:10:04.480 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:10:04.480 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:10:04.480 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:10:04.480 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:10:04.480 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:10:04.480 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:10:04.481 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:10:04.481 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2020-04-10T07:10:04.491 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:10:04.491 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:10:04.491 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:10:04.491 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test_clone_v2 2020-04-10T07:10:04.587 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:Invalid command: unused arguments: ['mirror/test_clone_v2'] 2020-04-10T07:10:04.587 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status : get status for rbd mirror 2020-04-10T07:10:04.587 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:10:04.588 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:3 rbd mirror status mirror/test_clone_v2 2020-04-10T07:10:04.588 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:3 2020-04-10T07:10:04.588 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:10:04.588 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:10:04.588 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:3 cluster instance 2020-04-10T07:10:04.588 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:3 2020-04-10T07:10:04.588 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:10:04.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:10:04.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:10:04.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:10:04.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 3 = cluster1:3 ] 2020-04-10T07:10:04.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:10:04.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:10:04.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=3 2020-04-10T07:10:04.590 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:10:04.590 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:3 cluster1 2020-04-10T07:10:04.590 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:3 2020-04-10T07:10:04.590 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:04.590 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:10:04.590 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:3 local_cluster instance 2020-04-10T07:10:04.590 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:3 2020-04-10T07:10:04.591 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:10:04.591 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:10:04.591 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:10:04.591 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:10:04.591 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 3 = cluster1:3 ] 2020-04-10T07:10:04.591 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:10:04.591 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:10:04.591 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=3 2020-04-10T07:10:04.592 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:10:04.592 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.3 admin socket 2020-04-10T07:10:04.604 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:10:04.605 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:10:04.605 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:10:04.605 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/test_clone_v2 2020-04-10T07:10:04.697 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:Invalid command: unused arguments: ['mirror/test_clone_v2'] 2020-04-10T07:10:04.697 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status : get status for rbd mirror 2020-04-10T07:10:04.698 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:10:04.698 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:10:05.694 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ cluster1 != cluster1 ] 2020-04-10T07:10:05.694 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:10:05.697 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test_clone_v2 2020-04-10T07:10:05.697 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:10:05.698 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:10:05.698 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:10:05.698 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:10:05.698 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:10:05.698 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:10:05.698 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:10:05.698 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:10:05.699 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:10:05.699 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:10:05.699 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:10:05.699 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:10:05.699 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:10:05.699 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:10:05.699 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster1 2020-04-10T07:10:05.700 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:10:05.700 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:05.700 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:10:05.700 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:10:05.700 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:10:05.700 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:10:05.700 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:10:05.702 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:10:05.703 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:10:05.703 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:10:05.703 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:10:05.704 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:10:05.704 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:10:05.704 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:10:05.704 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:10:05.718 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:10:05.719 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:10:05.719 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:10:05.719 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test_clone_v2 2020-04-10T07:10:05.814 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:Invalid command: unused arguments: ['mirror/test_clone_v2'] 2020-04-10T07:10:05.814 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status : get status for rbd mirror 2020-04-10T07:10:05.814 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:10:05.817 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test_clone_v2 2020-04-10T07:10:05.817 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:1 2020-04-10T07:10:05.817 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:10:05.817 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:10:05.817 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 cluster instance 2020-04-10T07:10:05.817 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:10:05.817 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:10:05.818 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:10:05.818 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:10:05.818 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:10:05.818 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:10:05.818 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:10:05.818 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:10:05.818 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:10:05.819 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:10:05.819 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:1 cluster1 2020-04-10T07:10:05.819 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:1 2020-04-10T07:10:05.819 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:05.819 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:10:05.819 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2020-04-10T07:10:05.819 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:10:05.820 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:10:05.820 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:10:05.820 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:10:05.820 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:10:05.820 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:10:05.820 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:10:05.821 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:10:05.821 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:10:05.821 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:10:05.821 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2020-04-10T07:10:05.840 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:10:05.840 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:10:05.840 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:10:05.840 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test_clone_v2 2020-04-10T07:10:05.930 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:Invalid command: unused arguments: ['mirror/test_clone_v2'] 2020-04-10T07:10:05.930 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status : get status for rbd mirror 2020-04-10T07:10:05.930 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:10:05.932 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:2 rbd mirror status mirror/test_clone_v2 2020-04-10T07:10:05.932 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:2 2020-04-10T07:10:05.932 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:10:05.932 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:10:05.932 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 cluster instance 2020-04-10T07:10:05.934 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:10:05.934 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:10:05.934 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:10:05.935 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:10:05.935 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:10:05.935 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:10:05.935 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:10:05.935 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:10:05.935 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:10:05.936 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:10:05.936 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:2 cluster1 2020-04-10T07:10:05.936 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:2 2020-04-10T07:10:05.936 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:05.936 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:10:05.936 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2020-04-10T07:10:05.937 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:10:05.937 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:10:05.937 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:10:05.937 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:10:05.937 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:10:05.937 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:10:05.937 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:10:05.938 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:10:05.938 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:10:05.938 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:10:05.938 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2020-04-10T07:10:05.953 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:10:05.953 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:10:05.953 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:10:05.953 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test_clone_v2 2020-04-10T07:10:06.040 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:Invalid command: unused arguments: ['mirror/test_clone_v2'] 2020-04-10T07:10:06.040 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status : get status for rbd mirror 2020-04-10T07:10:06.040 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:10:06.042 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:3 rbd mirror status mirror/test_clone_v2 2020-04-10T07:10:06.042 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:3 2020-04-10T07:10:06.043 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:10:06.043 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:10:06.043 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:3 cluster instance 2020-04-10T07:10:06.043 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:3 2020-04-10T07:10:06.043 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:10:06.043 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:10:06.043 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:10:06.044 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:10:06.044 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 3 = cluster1:3 ] 2020-04-10T07:10:06.044 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:10:06.044 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:10:06.044 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=3 2020-04-10T07:10:06.044 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:10:06.044 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:3 cluster1 2020-04-10T07:10:06.045 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:3 2020-04-10T07:10:06.045 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:06.045 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:10:06.045 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:3 local_cluster instance 2020-04-10T07:10:06.045 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:3 2020-04-10T07:10:06.045 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:10:06.046 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:10:06.046 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:10:06.046 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:10:06.046 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 3 = cluster1:3 ] 2020-04-10T07:10:06.046 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:10:06.046 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:10:06.046 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=3 2020-04-10T07:10:06.047 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:10:06.047 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.3 admin socket 2020-04-10T07:10:06.064 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:10:06.065 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:10:06.065 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:10:06.065 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/test_clone_v2 2020-04-10T07:10:06.172 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:10:06.174 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ status_result= "state": "Replaying" 2020-04-10T07:10:06.174 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo "state": "Replaying" 2020-04-10T07:10:06.174 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep -i Replaying 2020-04-10T07:10:06.174 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ current_state=started 2020-04-10T07:10:06.174 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test started = started 2020-04-10T07:10:06.175 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:10:06.175 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_clone_format cluster1 mirror test_clone_v2 2020-04-10T07:10:06.175 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:06.175 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:06.175 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_clone_v2 2020-04-10T07:10:06.175 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ awk BEGIN { 2020-04-10T07:10:06.175 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: format = 1 2020-04-10T07:10:06.176 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: } 2020-04-10T07:10:06.176 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: $1 == "parent:" { 2020-04-10T07:10:06.176 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: parent = $2 2020-04-10T07:10:06.176 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: } 2020-04-10T07:10:06.176 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: /op_features: .*clone-child/ { 2020-04-10T07:10:06.176 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: format = 2 2020-04-10T07:10:06.176 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: } 2020-04-10T07:10:06.177 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: END { 2020-04-10T07:10:06.177 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: if (!parent) exit 1 2020-04-10T07:10:06.177 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: print format 2020-04-10T07:10:06.177 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: } 2020-04-10T07:10:06.177 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 info mirror/test_clone_v2 2020-04-10T07:10:06.178 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "state": "Replaying" 2020-04-10T07:10:06.247 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test 2 = 2 2020-04-10T07:10:06.247 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ remove_snapshot cluster2 mirror_parent test_parent snap_v2 2020-04-10T07:10:06.247 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:06.248 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror_parent 2020-04-10T07:10:06.248 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_parent 2020-04-10T07:10:06.248 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snap=snap_v2 2020-04-10T07:10:06.248 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 snap rm mirror_parent/test_parent@snap_v2 2020-04-10T07:10:06.315 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: Removing snap: 100% complete...done. 2020-04-10T07:10:06.321 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mirror_image_snapshot cluster2 mirror_parent test_parent 2020-04-10T07:10:06.321 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:06.321 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror_parent 2020-04-10T07:10:06.322 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_parent 2020-04-10T07:10:06.322 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror_parent/test_parent 2020-04-10T07:10:07.183 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Snapshot ID: 11 2020-04-10T07:10:07.189 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_snap_moved_to_trash cluster2 mirror_parent test_parent snap_v2 2020-04-10T07:10:07.189 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:07.189 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror_parent 2020-04-10T07:10:07.189 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_parent 2020-04-10T07:10:07.189 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snap_name=snap_v2 2020-04-10T07:10:07.190 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 snap ls mirror_parent/test_parent --all 2020-04-10T07:10:07.190 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep -F trash (snap_v2) 2020-04-10T07:10:07.236 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 8 8c7945aa-c3db-4193-b6ba-c73d145583e2 128 MiB Fri Apr 10 07:10:01 2020 trash (snap_v2) 2020-04-10T07:10:07.236 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_snap_moved_to_trash cluster1 mirror_parent test_parent snap_v2 2020-04-10T07:10:07.236 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local s 2020-04-10T07:10:07.236 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:10:08.238 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_snap_moved_to_trash cluster1 mirror_parent test_parent snap_v2 2020-04-10T07:10:08.238 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:08.238 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror_parent 2020-04-10T07:10:08.238 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_parent 2020-04-10T07:10:08.238 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snap_name=snap_v2 2020-04-10T07:10:08.239 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ + rbd --cluster cluster1 snap ls mirror_parent/test_parentgrep -F --all trash (snap_v2) 2020-04-10T07:10:08.239 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: 2020-04-10T07:10:08.303 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 7 2914e994-c377-472d-9ac0-0f23bd39371c 128 MiB Fri Apr 10 07:10:03 2020 trash (snap_v2) 2020-04-10T07:10:08.303 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:10:08.304 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ remove_image_retry cluster2 mirror test_clone_v2 2020-04-10T07:10:08.304 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:08.304 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:08.304 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_clone_v2 2020-04-10T07:10:08.304 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0 2020-04-10T07:10:08.304 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ remove_image cluster2 mirror test_clone_v2 2020-04-10T07:10:08.305 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:08.305 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:08.305 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_clone_v2 2020-04-10T07:10:08.305 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster2 snap purge mirror/test_clone_v2 2020-04-10T07:10:08.383 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster2 rm mirror/test_clone_v2 2020-04-10T07:10:10.232 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2020-04-10T07:10:10.235 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:10:10.235 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_present cluster1 mirror test_clone_v2 deleted 2020-04-10T07:10:10.235 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:10.235 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:10.235 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_clone_v2 2020-04-10T07:10:10.236 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state=deleted 2020-04-10T07:10:10.236 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image_id= 2020-04-10T07:10:10.236 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local s 2020-04-10T07:10:10.236 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 2020-04-10T07:10:10.236 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_image_id cluster1 mirror test_clone_v2 2020-04-10T07:10:10.236 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:10.236 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:10.237 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_clone_v2 2020-04-10T07:10:10.237 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster1 info mirror/test_clone_v2 2020-04-10T07:10:10.237 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -ne s/^.*block_name_prefix: rbd_data\.//p 2020-04-10T07:10:10.268 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: error opening image test_clone_v2: (2) No such file or directory 2020-04-10T07:10:10.271 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ image_id= 2020-04-10T07:10:10.271 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0.1 2020-04-10T07:10:10.373 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_image_present cluster1 mirror test_clone_v2 deleted 2020-04-10T07:10:10.373 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:10.373 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:10.374 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_clone_v2 2020-04-10T07:10:10.374 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local test_state=deleted 2020-04-10T07:10:10.374 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image_id= 2020-04-10T07:10:10.374 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_state=deleted 2020-04-10T07:10:10.374 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_image_id 2020-04-10T07:10:10.374 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_image_id cluster1 mirror test_clone_v2 2020-04-10T07:10:10.375 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:10.375 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:10.375 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_clone_v2 2020-04-10T07:10:10.375 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster1 info mirror/test_clone_v2 2020-04-10T07:10:10.375 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -ne s/^.*block_name_prefix: rbd_data\.//p 2020-04-10T07:10:10.415 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: error opening image test_clone_v2: (2) No such file or directory 2020-04-10T07:10:10.419 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ current_image_id= 2020-04-10T07:10:10.419 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 2020-04-10T07:10:10.419 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test deleted = deleted 2020-04-10T07:10:10.419 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:10:10.419 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_snap_removed_from_trash cluster2 mirror_parent test_parent snap_v2 2020-04-10T07:10:10.419 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_snap_moved_to_trash cluster2 mirror_parent test_parent snap_v2 2020-04-10T07:10:10.420 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:10.420 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror_parent 2020-04-10T07:10:10.420 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_parent 2020-04-10T07:10:10.420 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snap_name=snap_v2 2020-04-10T07:10:10.420 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 snap ls mirror_parent/test_parent --all 2020-04-10T07:10:10.420 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep -F trash (snap_v2) 2020-04-10T07:10:10.478 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:10:10.478 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_snap_removed_from_trash cluster1 mirror_parent test_parent snap_v2 2020-04-10T07:10:10.478 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local s 2020-04-10T07:10:10.478 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:10:11.479 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_snap_removed_from_trash cluster1 mirror_parent test_parent snap_v2 2020-04-10T07:10:11.480 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_snap_moved_to_trash cluster1 mirror_parent test_parent snap_v2 2020-04-10T07:10:11.480 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:11.480 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror_parent 2020-04-10T07:10:11.480 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_parent 2020-04-10T07:10:11.480 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snap_name=snap_v2 2020-04-10T07:10:11.480 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 snap ls mirror_parent/test_parent --all 2020-04-10T07:10:11.481 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep -F trash (snap_v2) 2020-04-10T07:10:11.550 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:10:11.550 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:10:11.550 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ testlog - clone v2 non-primary 2020-04-10T07:10:11.550 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ tee -a /tmp/tmp.8SwBCsYPSC/rbd-mirror.test.log 2020-04-10T07:10:11.550 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ date +%F %T 2020-04-10T07:10:11.551 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10 07:10:11 - clone v2 non-primary 2020-04-10T07:10:11.552 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:2020-04-10 07:10:11 - clone v2 non-primary 2020-04-10T07:10:11.552 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ create_snapshot cluster2 mirror_parent test_parent snap_v2 2020-04-10T07:10:11.552 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:11.552 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror_parent 2020-04-10T07:10:11.552 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_parent 2020-04-10T07:10:11.552 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snap=snap_v2 2020-04-10T07:10:11.552 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 snap create mirror_parent/test_parent@snap_v2 2020-04-10T07:10:12.205 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mirror_image_snapshot cluster2 mirror_parent test_parent 2020-04-10T07:10:12.205 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:12.205 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror_parent 2020-04-10T07:10:12.205 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_parent 2020-04-10T07:10:12.206 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror_parent/test_parent 2020-04-10T07:10:13.220 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Snapshot ID: 15 2020-04-10T07:10:13.221 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_snap_present cluster1 mirror_parent test_parent snap_v2 2020-04-10T07:10:13.221 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:13.221 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror_parent 2020-04-10T07:10:13.221 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_parent 2020-04-10T07:10:13.222 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snap_name=snap_v2 2020-04-10T07:10:13.222 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local s 2020-04-10T07:10:13.222 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:10:14.227 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 info mirror_parent/test_parent@snap_v2 2020-04-10T07:10:14.282 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:rbd image 'test_parent': 2020-04-10T07:10:14.282 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: size 128 MiB in 32 objects 2020-04-10T07:10:14.282 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: order 22 (4 MiB objects) 2020-04-10T07:10:14.282 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: snapshot_count: 6 2020-04-10T07:10:14.282 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: id: 1154c59c0497 2020-04-10T07:10:14.283 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: block_name_prefix: rbd_data.1154c59c0497 2020-04-10T07:10:14.283 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: format: 2 2020-04-10T07:10:14.283 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: features: layering, exclusive-lock, journaling, non-primary 2020-04-10T07:10:14.283 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: op_features: 2020-04-10T07:10:14.283 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: flags: 2020-04-10T07:10:14.283 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: create_timestamp: Fri Apr 10 07:09:45 2020 2020-04-10T07:10:14.283 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: access_timestamp: Fri Apr 10 07:09:45 2020 2020-04-10T07:10:14.284 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: modify_timestamp: Fri Apr 10 07:09:45 2020 2020-04-10T07:10:14.284 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: protected: False 2020-04-10T07:10:14.284 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: journal: 1154c59c0497 2020-04-10T07:10:14.284 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring state: enabled 2020-04-10T07:10:14.284 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring mode: snapshot 2020-04-10T07:10:14.284 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring global id: dd523e97-79dd-49a7-b2c9-df255e9660d4 2020-04-10T07:10:14.284 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring primary: false 2020-04-10T07:10:14.288 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:10:14.288 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ clone_image_and_enable_mirror cluster1 mirror_parent test_parent snap_v2 mirror test_clone_v2 snapshot --rbd-default-clone-format 2 2020-04-10T07:10:14.288 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:14.288 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local parent_pool=mirror_parent 2020-04-10T07:10:14.288 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local parent_image=test_parent 2020-04-10T07:10:14.289 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local parent_snap=snap_v2 2020-04-10T07:10:14.289 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local clone_pool=mirror 2020-04-10T07:10:14.289 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local clone_image=test_clone_v2 2020-04-10T07:10:14.289 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 6 2020-04-10T07:10:14.289 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local mode=snapshot 2020-04-10T07:10:14.289 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n snapshot ] 2020-04-10T07:10:14.289 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:10:14.290 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ clone_image cluster1 mirror_parent test_parent snap_v2 mirror test_clone_v2 --rbd-default-clone-format 2 2020-04-10T07:10:14.290 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:14.293 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local parent_pool=mirror_parent 2020-04-10T07:10:14.293 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local parent_image=test_parent 2020-04-10T07:10:14.294 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local parent_snap=snap_v2 2020-04-10T07:10:14.294 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local clone_pool=mirror 2020-04-10T07:10:14.294 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local clone_image=test_clone_v2 2020-04-10T07:10:14.295 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 6 2020-04-10T07:10:14.295 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 clone mirror_parent/test_parent@snap_v2 mirror/test_clone_v2 --image-feature layering,exclusive-lock,journaling --rbd-default-clone-format 2 2020-04-10T07:10:14.415 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ enable_mirror cluster1 mirror test_clone_v2 snapshot 2020-04-10T07:10:14.415 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:14.415 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:14.415 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_clone_v2 2020-04-10T07:10:14.415 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local mode=snapshot 2020-04-10T07:10:14.415 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/test_clone_v2 snapshot 2020-04-10T07:10:15.086 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Mirroring enabled 2020-04-10T07:10:15.090 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ remove_snapshot cluster2 mirror_parent test_parent snap_v2 2020-04-10T07:10:15.090 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:15.090 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror_parent 2020-04-10T07:10:15.090 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_parent 2020-04-10T07:10:15.090 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snap=snap_v2 2020-04-10T07:10:15.091 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 snap rm mirror_parent/test_parent@snap_v2 2020-04-10T07:10:15.249 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: Removing snap: 100% complete...done. 2020-04-10T07:10:15.255 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_snap_removed_from_trash cluster2 mirror_parent test_parent snap_v2 2020-04-10T07:10:15.255 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_snap_moved_to_trash cluster2 mirror_parent test_parent snap_v2 2020-04-10T07:10:15.255 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:15.255 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror_parent 2020-04-10T07:10:15.255 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_parent 2020-04-10T07:10:15.256 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snap_name=snap_v2 2020-04-10T07:10:15.256 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 snap ls mirror_parent/test_parent --all 2020-04-10T07:10:15.256 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep -F trash (snap_v2) 2020-04-10T07:10:15.298 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:10:15.298 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mirror_image_snapshot cluster2 mirror_parent test_parent 2020-04-10T07:10:15.298 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:15.299 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror_parent 2020-04-10T07:10:15.299 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_parent 2020-04-10T07:10:15.299 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror_parent/test_parent 2020-04-10T07:10:16.142 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Snapshot ID: 18 2020-04-10T07:10:16.147 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_snap_moved_to_trash cluster1 mirror_parent test_parent snap_v2 2020-04-10T07:10:16.147 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local s 2020-04-10T07:10:16.147 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:10:17.148 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_snap_moved_to_trash cluster1 mirror_parent test_parent snap_v2 2020-04-10T07:10:17.148 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:17.149 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror_parent 2020-04-10T07:10:17.149 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_parent 2020-04-10T07:10:17.149 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snap_name=snap_v2 2020-04-10T07:10:17.149 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 snap ls mirror_parent/test_parent --all 2020-04-10T07:10:17.149 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep -F trash (snap_v2) 2020-04-10T07:10:17.202 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 11 7c804667-22e8-4144-8331-555a83f90999 128 MiB Fri Apr 10 07:10:14 2020 trash (snap_v2) 2020-04-10T07:10:17.202 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:10:17.202 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ remove_image_retry cluster1 mirror test_clone_v2 2020-04-10T07:10:17.202 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:17.202 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:17.202 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_clone_v2 2020-04-10T07:10:17.203 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0 2020-04-10T07:10:17.203 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ remove_image cluster1 mirror test_clone_v2 2020-04-10T07:10:17.203 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:17.204 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:17.204 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_clone_v2 2020-04-10T07:10:17.204 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster1 snap purge mirror/test_clone_v2 2020-04-10T07:10:17.303 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster1 rm mirror/test_clone_v2 2020-04-10T07:10:19.265 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2020-04-10T07:10:19.266 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:10:19.266 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_snap_removed_from_trash cluster1 mirror_parent test_parent snap_v2 2020-04-10T07:10:19.266 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local s 2020-04-10T07:10:19.266 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:10:20.268 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_snap_removed_from_trash cluster1 mirror_parent test_parent snap_v2 2020-04-10T07:10:20.268 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_snap_moved_to_trash cluster1 mirror_parent test_parent snap_v2 2020-04-10T07:10:20.268 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:20.269 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror_parent 2020-04-10T07:10:20.269 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_parent 2020-04-10T07:10:20.269 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snap_name=snap_v2 2020-04-10T07:10:20.269 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 snap ls mirror_parent/test_parent --all 2020-04-10T07:10:20.269 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep -F trash (snap_v2) 2020-04-10T07:10:20.335 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:10:20.335 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:10:20.336 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ testlog TEST: data pool 2020-04-10T07:10:20.336 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ tee -a /tmp/tmp.8SwBCsYPSC/rbd-mirror.test.log 2020-04-10T07:10:20.336 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ date +%F %T 2020-04-10T07:10:20.338 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10 07:10:20 TEST: data pool 2020-04-10T07:10:20.338 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:2020-04-10 07:10:20 TEST: data pool 2020-04-10T07:10:20.338 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ dp_image=test_data_pool 2020-04-10T07:10:20.339 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ create_image_and_enable_mirror cluster2 mirror test_data_pool snapshot 128 --data-pool mirror_parent 2020-04-10T07:10:20.339 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:20.339 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:10:20.339 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:20.339 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:10:20.339 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_data_pool 2020-04-10T07:10:20.339 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:10:20.340 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local mode=snapshot 2020-04-10T07:10:20.340 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n snapshot ] 2020-04-10T07:10:20.340 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:10:20.340 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ create_image cluster2 mirror test_data_pool 128 --data-pool mirror_parent 2020-04-10T07:10:20.340 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:20.340 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:10:20.340 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:20.341 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:10:20.341 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_data_pool 2020-04-10T07:10:20.341 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:10:20.341 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local size=128 2020-04-10T07:10:20.341 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n 128 ] 2020-04-10T07:10:20.341 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ size=128 2020-04-10T07:10:20.341 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:10:20.343 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 create --size 128 --image-feature layering,exclusive-lock,journaling --data-pool mirror_parent mirror/test_data_pool 2020-04-10T07:10:20.408 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ enable_mirror cluster2 mirror test_data_pool snapshot 2020-04-10T07:10:20.408 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:20.408 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:20.409 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_data_pool 2020-04-10T07:10:20.409 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local mode=snapshot 2020-04-10T07:10:20.409 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/test_data_pool snapshot 2020-04-10T07:10:20.988 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Mirroring enabled 2020-04-10T07:10:20.995 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_image_data_pool cluster2 mirror test_data_pool 2020-04-10T07:10:20.996 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:20.996 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:20.996 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_data_pool 2020-04-10T07:10:20.996 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 info mirror/test_data_pool 2020-04-10T07:10:20.996 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ awk $1 == "data_pool:" {print $2} 2020-04-10T07:10:21.047 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ data_pool=mirror_parent 2020-04-10T07:10:21.048 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test mirror_parent = mirror_parent 2020-04-10T07:10:21.048 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_started cluster1 mirror test_data_pool 2020-04-10T07:10:21.048 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:21.048 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:21.048 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_data_pool 2020-04-10T07:10:21.048 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_replay_state cluster1 mirror test_data_pool started 2020-04-10T07:10:21.049 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:21.049 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:21.049 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_data_pool 2020-04-10T07:10:21.049 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state=started 2020-04-10T07:10:21.049 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local s 2020-04-10T07:10:21.049 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:10:22.048 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_image_replay_state cluster1 mirror test_data_pool started 2020-04-10T07:10:22.048 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:22.048 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:22.048 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_data_pool 2020-04-10T07:10:22.049 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local test_state=started 2020-04-10T07:10:22.049 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_result 2020-04-10T07:10:22.049 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_state=stopped 2020-04-10T07:10:22.050 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test_data_pool 2020-04-10T07:10:22.050 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ + local cluster_instance=cluster1 2020-04-10T07:10:22.050 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:grep -i state 2020-04-10T07:10:22.050 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:10:22.051 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:22.051 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance=cluster1 2020-04-10T07:10:22.051 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local loop_instance 2020-04-10T07:10:22.051 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0 2020-04-10T07:10:22.051 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ cluster1 != cluster1 ] 2020-04-10T07:10:22.051 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:10:22.053 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test_data_pool 2020-04-10T07:10:22.053 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:10:22.053 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:10:22.053 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:10:22.053 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:10:22.054 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:10:22.054 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:10:22.054 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:10:22.054 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:10:22.054 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:10:22.054 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:10:22.054 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:10:22.055 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:10:22.055 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:10:22.055 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:10:22.055 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster1 2020-04-10T07:10:22.055 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:10:22.055 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:22.055 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:10:22.056 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:10:22.056 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:10:22.056 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:10:22.056 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:10:22.057 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:10:22.057 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:10:22.057 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:10:22.057 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:10:22.057 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:10:22.058 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:10:22.058 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:10:22.058 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:10:22.073 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:10:22.074 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:10:22.074 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:10:22.074 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test_data_pool 2020-04-10T07:10:22.173 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:Invalid command: unused arguments: ['mirror/test_data_pool'] 2020-04-10T07:10:22.173 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status : get status for rbd mirror 2020-04-10T07:10:22.173 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:10:22.176 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test_data_pool 2020-04-10T07:10:22.176 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:1 2020-04-10T07:10:22.176 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:10:22.176 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:10:22.176 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 cluster instance 2020-04-10T07:10:22.176 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:10:22.176 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:10:22.177 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:10:22.177 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:10:22.177 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:10:22.177 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:10:22.177 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:10:22.177 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:10:22.177 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:10:22.178 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:10:22.178 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:1 cluster1 2020-04-10T07:10:22.178 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:1 2020-04-10T07:10:22.178 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:22.178 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:10:22.178 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2020-04-10T07:10:22.179 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:10:22.179 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:10:22.179 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:10:22.179 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:10:22.179 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:10:22.179 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:10:22.180 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:10:22.180 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:10:22.180 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:10:22.180 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:10:22.180 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2020-04-10T07:10:22.194 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:10:22.194 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:10:22.194 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:10:22.194 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test_data_pool 2020-04-10T07:10:22.294 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:Invalid command: unused arguments: ['mirror/test_data_pool'] 2020-04-10T07:10:22.294 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status : get status for rbd mirror 2020-04-10T07:10:22.294 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:10:22.294 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:2 rbd mirror status mirror/test_data_pool 2020-04-10T07:10:22.294 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:2 2020-04-10T07:10:22.294 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:10:22.294 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:10:22.295 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 cluster instance 2020-04-10T07:10:22.295 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:10:22.295 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:10:22.295 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:10:22.295 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:10:22.295 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:10:22.295 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:10:22.296 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:10:22.296 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:10:22.296 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:10:22.296 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:10:22.296 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:2 cluster1 2020-04-10T07:10:22.296 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:2 2020-04-10T07:10:22.296 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:22.297 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:10:22.297 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2020-04-10T07:10:22.297 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:10:22.297 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:10:22.297 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:10:22.297 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:10:22.297 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:10:22.297 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:10:22.298 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:10:22.298 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:10:22.298 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:10:22.298 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:10:22.298 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2020-04-10T07:10:22.305 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:10:22.306 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:10:22.306 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:10:22.306 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test_data_pool 2020-04-10T07:10:22.397 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:Invalid command: unused arguments: ['mirror/test_data_pool'] 2020-04-10T07:10:22.398 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status : get status for rbd mirror 2020-04-10T07:10:22.398 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:10:22.400 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:3 rbd mirror status mirror/test_data_pool 2020-04-10T07:10:22.400 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:3 2020-04-10T07:10:22.400 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:10:22.400 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:10:22.401 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:3 cluster instance 2020-04-10T07:10:22.401 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:3 2020-04-10T07:10:22.401 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:10:22.401 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:10:22.401 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:10:22.401 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:10:22.401 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 3 = cluster1:3 ] 2020-04-10T07:10:22.402 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:10:22.402 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:10:22.402 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=3 2020-04-10T07:10:22.402 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:10:22.402 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:3 cluster1 2020-04-10T07:10:22.402 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:3 2020-04-10T07:10:22.402 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:22.402 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:10:22.403 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:3 local_cluster instance 2020-04-10T07:10:22.403 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:3 2020-04-10T07:10:22.403 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:10:22.403 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:10:22.403 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:10:22.403 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:10:22.403 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 3 = cluster1:3 ] 2020-04-10T07:10:22.404 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:10:22.404 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:10:22.404 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=3 2020-04-10T07:10:22.404 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:10:22.404 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.3 admin socket 2020-04-10T07:10:22.420 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:10:22.421 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:10:22.421 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:10:22.421 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/test_data_pool 2020-04-10T07:10:22.508 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:Invalid command: unused arguments: ['mirror/test_data_pool'] 2020-04-10T07:10:22.508 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd mirror status : get status for rbd mirror 2020-04-10T07:10:22.509 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:admin_socket: invalid command 2020-04-10T07:10:22.509 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:10:23.516 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ cluster1 != cluster1 ] 2020-04-10T07:10:23.517 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:10:23.517 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test_data_pool 2020-04-10T07:10:23.517 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:10:23.517 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:10:23.517 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:10:23.517 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:10:23.517 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:10:23.518 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:10:23.518 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:10:23.518 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:10:23.518 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:10:23.518 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:10:23.518 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:10:23.518 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:10:23.519 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:10:23.519 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:10:23.519 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster1 2020-04-10T07:10:23.519 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:10:23.519 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:23.519 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:10:23.519 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:10:23.520 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:10:23.520 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:10:23.520 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:10:23.520 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:10:23.520 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:10:23.520 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:10:23.520 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:10:23.520 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:10:23.521 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:10:23.521 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:10:23.521 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:10:23.537 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:10:23.537 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:10:23.537 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:10:23.537 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test_data_pool 2020-04-10T07:10:23.644 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:10:23.645 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ status_result= "state": "Replaying" 2020-04-10T07:10:23.646 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo "state": "Replaying" 2020-04-10T07:10:23.646 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep -i Replaying 2020-04-10T07:10:23.648 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "state": "Replaying" 2020-04-10T07:10:23.648 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ current_state=started 2020-04-10T07:10:23.649 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test started = started 2020-04-10T07:10:23.649 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:10:23.649 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_image_data_pool cluster1 mirror test_data_pool 2020-04-10T07:10:23.649 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:23.649 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:23.649 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_data_pool 2020-04-10T07:10:23.650 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 info mirror/test_data_pool 2020-04-10T07:10:23.650 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ awk $1 == "data_pool:" {print $2} 2020-04-10T07:10:23.707 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ data_pool=mirror_parent 2020-04-10T07:10:23.707 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test mirror_parent = mirror_parent 2020-04-10T07:10:23.707 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ create_snapshot cluster2 mirror test_data_pool snap1 2020-04-10T07:10:23.708 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:23.708 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:23.708 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_data_pool 2020-04-10T07:10:23.708 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snap=snap1 2020-04-10T07:10:23.708 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 snap create mirror/test_data_pool@snap1 2020-04-10T07:10:24.140 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ write_image cluster2 mirror test_data_pool 100 2020-04-10T07:10:24.141 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:24.141 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:24.141 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_data_pool 2020-04-10T07:10:24.141 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local count=100 2020-04-10T07:10:24.141 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local size= 2020-04-10T07:10:24.141 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 2020-04-10T07:10:24.141 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ size=4096 2020-04-10T07:10:24.142 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 bench mirror/test_data_pool --io-type write --io-size 4096 --io-threads 1 --io-total 409600 --io-pattern rand 2020-04-10T07:10:24.210 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:bench type write io_size 4096 io_threads 1 bytes 409600 pattern random 2020-04-10T07:10:24.274 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: SEC OPS OPS/SEC BYTES/SEC 2020-04-10T07:10:24.274 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:elapsed: 0 ops: 100 ops/sec: 1562.5 bytes/sec: 6.1 MiB/s 2020-04-10T07:10:24.285 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ create_snapshot cluster2 mirror test_data_pool snap2 2020-04-10T07:10:24.285 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:24.285 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:24.285 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_data_pool 2020-04-10T07:10:24.286 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snap=snap2 2020-04-10T07:10:24.286 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 snap create mirror/test_data_pool@snap2 2020-04-10T07:10:25.149 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ write_image cluster2 mirror test_data_pool 100 2020-04-10T07:10:25.150 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:25.150 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:25.150 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_data_pool 2020-04-10T07:10:25.150 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local count=100 2020-04-10T07:10:25.150 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local size= 2020-04-10T07:10:25.150 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 2020-04-10T07:10:25.151 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ size=4096 2020-04-10T07:10:25.151 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 bench mirror/test_data_pool --io-type write --io-size 4096 --io-threads 1 --io-total 409600 --io-pattern rand 2020-04-10T07:10:25.225 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:bench type write io_size 4096 io_threads 1 bytes 409600 pattern random 2020-04-10T07:10:25.295 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: SEC OPS OPS/SEC BYTES/SEC 2020-04-10T07:10:25.296 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:elapsed: 0 ops: 100 ops/sec: 1470.59 bytes/sec: 5.7 MiB/s 2020-04-10T07:10:25.308 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror test_data_pool 2020-04-10T07:10:25.308 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1 2020-04-10T07:10:25.308 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:25.308 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:25.308 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_data_pool 2020-04-10T07:10:25.309 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ snapshot = journal ] 2020-04-10T07:10:25.309 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ snapshot = snapshot ] 2020-04-10T07:10:25.309 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror test_data_pool 2020-04-10T07:10:25.309 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1 2020-04-10T07:10:25.309 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:25.309 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:25.309 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_data_pool 2020-04-10T07:10:25.309 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster2-mirror-test_data_pool.status 2020-04-10T07:10:25.310 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2-mirror-test_data_pool.status 2020-04-10T07:10:25.310 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:10:25.311 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster2-mirror-test_data_pool.status 2020-04-10T07:10:25.311 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror-test_data_pool.status 2020-04-10T07:10:25.311 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror-test_data_pool.status 2020-04-10T07:10:25.311 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:10:25.312 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_data_pool.status 2020-04-10T07:10:25.312 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mirror_image_snapshot cluster2 mirror test_data_pool 2020-04-10T07:10:25.312 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:25.312 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:25.312 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_data_pool 2020-04-10T07:10:25.313 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test_data_pool 2020-04-10T07:10:26.153 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Snapshot ID: 23 2020-04-10T07:10:26.163 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_newest_mirror_snapshot cluster2 mirror test_data_pool /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test_data_pool.status 2020-04-10T07:10:26.164 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:26.164 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:26.164 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_data_pool 2020-04-10T07:10:26.164 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local log=/tmp/tmp.8SwBCsYPSC/cluster2-mirror-test_data_pool.status 2020-04-10T07:10:26.164 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 snap list --all mirror/test_data_pool --format xml 2020-04-10T07:10:26.164 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2020-04-10T07:10:26.211 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //snapshot/id 2020-04-10T07:10:26.214 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snapshot_id=23 2020-04-10T07:10:26.215 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ true 2020-04-10T07:10:26.215 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0.2 2020-04-10T07:10:26.416 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_newest_mirror_snapshot cluster1 mirror test_data_pool /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_data_pool.status 2020-04-10T07:10:26.417 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:26.417 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:26.417 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_data_pool 2020-04-10T07:10:26.417 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_data_pool.status 2020-04-10T07:10:26.417 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 snap list --all mirror/test_data_pool --format xml 2020-04-10T07:10:26.417 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2020-04-10T07:10:26.484 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2020-04-10T07:10:26.489 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local primary_snapshot_id=20 2020-04-10T07:10:26.489 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test 23 = 20 2020-04-10T07:10:26.490 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0.4 2020-04-10T07:10:26.891 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_newest_mirror_snapshot cluster1 mirror test_data_pool /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_data_pool.status 2020-04-10T07:10:26.891 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:26.892 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:26.892 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_data_pool 2020-04-10T07:10:26.892 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_data_pool.status 2020-04-10T07:10:26.892 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 snap list --all mirror/test_data_pool --format xml 2020-04-10T07:10:26.892 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2020-04-10T07:10:26.959 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2020-04-10T07:10:26.964 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local primary_snapshot_id=20 2020-04-10T07:10:26.964 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test 23 = 20 2020-04-10T07:10:26.964 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0.8 2020-04-10T07:10:27.766 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_newest_mirror_snapshot cluster1 mirror test_data_pool /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_data_pool.status 2020-04-10T07:10:27.766 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:27.766 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:27.767 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_data_pool 2020-04-10T07:10:27.767 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_data_pool.status 2020-04-10T07:10:27.767 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 snap list --all mirror/test_data_pool --format xml 2020-04-10T07:10:27.767 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2020-04-10T07:10:27.837 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ true 2020-04-10T07:10:27.837 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2020-04-10T07:10:27.841 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:-:1.1: Document is empty 2020-04-10T07:10:27.841 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: 2020-04-10T07:10:27.841 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:^ 2020-04-10T07:10:27.842 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local primary_snapshot_id= 2020-04-10T07:10:27.842 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test 23 = 2020-04-10T07:10:27.842 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1.6 2020-04-10T07:10:29.443 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_newest_mirror_snapshot cluster1 mirror test_data_pool /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_data_pool.status 2020-04-10T07:10:29.443 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:29.444 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:29.444 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_data_pool 2020-04-10T07:10:29.444 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_data_pool.status 2020-04-10T07:10:29.444 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 snap list --all mirror/test_data_pool --format xml 2020-04-10T07:10:29.444 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2020-04-10T07:10:29.509 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2020-04-10T07:10:29.514 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local primary_snapshot_id=23 2020-04-10T07:10:29.514 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test 23 = 23 2020-04-10T07:10:29.514 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:10:29.514 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test_data_pool up+replaying 2020-04-10T07:10:29.514 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:29.515 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:29.515 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_data_pool 2020-04-10T07:10:29.515 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:10:29.515 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:10:29.515 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:10:29.515 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:10:30.516 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_status_in_pool_dir cluster1 mirror test_data_pool up+replaying 2020-04-10T07:10:30.516 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:30.516 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:30.516 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_data_pool 2020-04-10T07:10:30.516 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:10:30.516 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local description_pattern= 2020-04-10T07:10:30.517 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local service_pattern= 2020-04-10T07:10:30.517 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror-test_data_pool.mirror_status 2020-04-10T07:10:30.517 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror-test_data_pool.mirror_status 2020-04-10T07:10:30.517 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:10:30.533 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_data_pool.mirror_status 2020-04-10T07:10:30.533 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test_data_pool 2020-04-10T07:10:30.534 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ tee /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_data_pool.mirror_status 2020-04-10T07:10:30.587 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:test_data_pool: 2020-04-10T07:10:30.588 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: global_id: a8183526-4331-4f0d-af4a-7c93c501e927 2020-04-10T07:10:30.588 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+replaying 2020-04-10T07:10:30.588 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: replaying, {"replay_state": "idle", "remote_snapshot_timestamp": 1586502620} 2020-04-10T07:10:30.588 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: service: mirror.0 on smithi204 2020-04-10T07:10:30.588 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:10:23 2020-04-10T07:10:30.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: peer_sites: 2020-04-10T07:10:30.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: name: cluster2-DNE 2020-04-10T07:10:30.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: state: up+stopped 2020-04-10T07:10:30.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: description: local image is primary 2020-04-10T07:10:30.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: last_update: 2020-04-10 07:10:26 2020-04-10T07:10:30.595 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ state: .*up+replaying /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_data_pool.mirror_status 2020-04-10T07:10:30.604 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+replaying 2020-04-10T07:10:30.604 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: replaying, {"replay_state": "idle", "remote_snapshot_timestamp": 1586502620} 2020-04-10T07:10:30.605 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+replaying 2020-04-10T07:10:30.605 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^ description: .* /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_data_pool.mirror_status 2020-04-10T07:10:30.605 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:10:30.605 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+replaying 2020-04-10T07:10:30.605 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep ^up+ 2020-04-10T07:10:30.608 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep service: *mirror..* on /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_data_pool.mirror_status 2020-04-10T07:10:30.608 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_data_pool.mirror_status 2020-04-10T07:10:30.608 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update=2020-04-10 07:10:23 2020-04-10T07:10:30.608 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test_data_pool up+replaying 2020-04-10 07:10:23 2020-04-10T07:10:30.608 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:30.609 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:30.611 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_data_pool 2020-04-10T07:10:30.611 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state_pattern=up+replaying 2020-04-10T07:10:30.611 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local prev_last_update=2020-04-10 07:10:23 2020-04-10T07:10:30.611 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror.mirror_status 2020-04-10T07:10:30.611 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror.mirror_status 2020-04-10T07:10:30.612 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:10:30.612 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local status_log=/tmp/tmp.8SwBCsYPSC/cluster1-mirror.mirror_status 2020-04-10T07:10:30.612 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2020-04-10T07:10:30.612 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: service: mirror.0 on smithi204 2020-04-10T07:10:30.653 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: failed to query services: (13) Permission denied 2020-04-10T07:10:30.736 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local last_update state 2020-04-10T07:10:30.736 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test_data_pool']/last_update 2020-04-10T07:10:30.740 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ last_update=2020-04-10 07:10:23 2020-04-10T07:10:30.741 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ xmlstarlet sel -t -v //images/image[name='test_data_pool']/state 2020-04-10T07:10:30.745 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ state=up+replaying 2020-04-10T07:10:30.745 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo up+replaying 2020-04-10T07:10:30.746 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ grep up+replaying 2020-04-10T07:10:30.746 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:up+replaying 2020-04-10T07:10:30.747 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:10:30.748 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:10:30.748 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ compare_images mirror test_data_pool@snap1 2020-04-10T07:10:30.748 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:30.748 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_data_pool@snap1 2020-04-10T07:10:30.749 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local ret=0 2020-04-10T07:10:30.749 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster2-mirror-test_data_pool@snap1.export 2020-04-10T07:10:30.749 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2-mirror-test_data_pool@snap1.export 2020-04-10T07:10:30.749 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:10:30.750 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local rmt_export=/tmp/tmp.8SwBCsYPSC/cluster2-mirror-test_data_pool@snap1.export 2020-04-10T07:10:30.751 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror-test_data_pool@snap1.export 2020-04-10T07:10:30.751 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror-test_data_pool@snap1.export 2020-04-10T07:10:30.751 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:10:30.754 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local loc_export=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_data_pool@snap1.export 2020-04-10T07:10:30.754 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rm -f /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test_data_pool@snap1.export /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_data_pool@snap1.export 2020-04-10T07:10:30.755 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 export mirror/test_data_pool@snap1 /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test_data_pool@snap1.export 2020-04-10T07:10:30.862 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2020-04-10T07:10:30.865 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 export mirror/test_data_pool@snap1 /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_data_pool@snap1.export 2020-04-10T07:10:30.973 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2020-04-10T07:10:30.977 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cmp /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test_data_pool@snap1.export /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_data_pool@snap1.export 2020-04-10T07:10:31.085 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rm -f /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test_data_pool@snap1.export /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_data_pool@snap1.export 2020-04-10T07:10:31.110 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:10:31.110 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ compare_images mirror test_data_pool@snap2 2020-04-10T07:10:31.110 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:31.110 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_data_pool@snap2 2020-04-10T07:10:31.111 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local ret=0 2020-04-10T07:10:31.111 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster2-mirror-test_data_pool@snap2.export 2020-04-10T07:10:31.111 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2-mirror-test_data_pool@snap2.export 2020-04-10T07:10:31.111 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:10:31.112 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local rmt_export=/tmp/tmp.8SwBCsYPSC/cluster2-mirror-test_data_pool@snap2.export 2020-04-10T07:10:31.112 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror-test_data_pool@snap2.export 2020-04-10T07:10:31.112 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo+ cluster1-mirror-test_data_pool@snap2.export 2020-04-10T07:10:31.113 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:sed -e s|[/ ]|_|g 2020-04-10T07:10:31.113 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local loc_export=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_data_pool@snap2.export 2020-04-10T07:10:31.113 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rm -f /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test_data_pool@snap2.export /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_data_pool@snap2.export 2020-04-10T07:10:31.113 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 export mirror/test_data_pool@snap2 /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test_data_pool@snap2.export 2020-04-10T07:10:31.270 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2020-04-10T07:10:31.273 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 export mirror/test_data_pool@snap2 /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_data_pool@snap2.export 2020-04-10T07:10:31.458 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2020-04-10T07:10:31.461 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cmp /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test_data_pool@snap2.export /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_data_pool@snap2.export 2020-04-10T07:10:31.535 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rm -f /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test_data_pool@snap2.export /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_data_pool@snap2.export 2020-04-10T07:10:31.574 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:10:31.574 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ compare_images mirror test_data_pool 2020-04-10T07:10:31.574 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:31.574 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test_data_pool 2020-04-10T07:10:31.575 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local ret=0 2020-04-10T07:10:31.575 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster2-mirror-test_data_pool.export 2020-04-10T07:10:31.575 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2-mirror-test_data_pool.export 2020-04-10T07:10:31.575 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:10:31.576 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local rmt_export=/tmp/tmp.8SwBCsYPSC/cluster2-mirror-test_data_pool.export 2020-04-10T07:10:31.576 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mkfname cluster1-mirror-test_data_pool.export 2020-04-10T07:10:31.576 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1-mirror-test_data_pool.export 2020-04-10T07:10:31.576 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -e s|[/ ]|_|g 2020-04-10T07:10:31.577 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local loc_export=/tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_data_pool.export 2020-04-10T07:10:31.577 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rm -f /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test_data_pool.export /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_data_pool.export 2020-04-10T07:10:31.577 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 export mirror/test_data_pool /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test_data_pool.export 2020-04-10T07:10:31.757 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2020-04-10T07:10:31.761 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 export mirror/test_data_pool /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_data_pool.export 2020-04-10T07:10:31.955 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2020-04-10T07:10:31.958 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cmp /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test_data_pool.export /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_data_pool.export 2020-04-10T07:10:32.030 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rm -f /tmp/tmp.8SwBCsYPSC/cluster2-mirror-test_data_pool.export /tmp/tmp.8SwBCsYPSC/cluster1-mirror-test_data_pool.export 2020-04-10T07:10:32.069 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:10:32.069 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ testlog TEST: disable mirroring / delete non-primary image 2020-04-10T07:10:32.069 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ tee -a /tmp/tmp.8SwBCsYPSC/rbd-mirror.test.log 2020-04-10T07:10:32.070 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ date +%F %T 2020-04-10T07:10:32.073 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10 07:10:32 TEST: disable mirroring / delete non-primary image 2020-04-10T07:10:32.073 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:2020-04-10 07:10:32 TEST: disable mirroring / delete non-primary image 2020-04-10T07:10:32.073 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ image2=test2 2020-04-10T07:10:32.073 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ image3=test3 2020-04-10T07:10:32.074 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ image4=test4 2020-04-10T07:10:32.074 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ image5=test5 2020-04-10T07:10:32.074 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ create_image_and_enable_mirror cluster2 mirror test2 2020-04-10T07:10:32.074 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:32.074 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:10:32.074 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:32.074 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:10:32.075 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test2 2020-04-10T07:10:32.075 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:10:32.075 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local mode=snapshot 2020-04-10T07:10:32.075 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:10:32.075 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ create_image cluster2 mirror test2 2020-04-10T07:10:32.075 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:32.075 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:10:32.075 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:32.076 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:10:32.076 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test2 2020-04-10T07:10:32.076 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:10:32.076 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local size=128 2020-04-10T07:10:32.076 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:10:32.076 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 create --size 128 --image-feature layering,exclusive-lock,journaling mirror/test2 2020-04-10T07:10:32.343 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ enable_mirror cluster2 mirror test2 snapshot 2020-04-10T07:10:32.343 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:32.343 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:32.344 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test2 2020-04-10T07:10:32.344 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local mode=snapshot 2020-04-10T07:10:32.344 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/test2 snapshot 2020-04-10T07:10:33.180 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Mirroring enabled 2020-04-10T07:10:33.190 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ write_image cluster2 mirror test2 100 2020-04-10T07:10:33.190 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:33.191 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:33.191 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test2 2020-04-10T07:10:33.191 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local count=100 2020-04-10T07:10:33.191 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local size= 2020-04-10T07:10:33.191 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 2020-04-10T07:10:33.191 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ size=4096 2020-04-10T07:10:33.192 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 bench mirror/test2 --io-type write --io-size 4096 --io-threads 1 --io-total 409600 --io-pattern rand 2020-04-10T07:10:33.258 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:bench type write io_size 4096 io_threads 1 bytes 409600 pattern random 2020-04-10T07:10:33.325 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: SEC OPS OPS/SEC BYTES/SEC 2020-04-10T07:10:33.326 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:elapsed: 0 ops: 100 ops/sec: 1470.83 bytes/sec: 5.7 MiB/s 2020-04-10T07:10:33.337 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ create_snapshot cluster2 mirror test2 snap1 2020-04-10T07:10:33.337 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:33.338 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:33.338 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test2 2020-04-10T07:10:33.338 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snap=snap1 2020-04-10T07:10:33.338 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 snap create mirror/test2@snap1 2020-04-10T07:10:34.188 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ create_snapshot cluster2 mirror test2 snap2 2020-04-10T07:10:34.189 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:34.189 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:34.189 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test2 2020-04-10T07:10:34.189 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snap=snap2 2020-04-10T07:10:34.189 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 snap create mirror/test2@snap2 2020-04-10T07:10:35.196 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ test2 = test4 ] 2020-04-10T07:10:35.196 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ test2 = test5 ] 2020-04-10T07:10:35.196 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ write_image cluster2 mirror test2 100 2020-04-10T07:10:35.196 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:35.197 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:35.197 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test2 2020-04-10T07:10:35.197 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local count=100 2020-04-10T07:10:35.197 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local size= 2020-04-10T07:10:35.197 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 2020-04-10T07:10:35.197 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ size=4096 2020-04-10T07:10:35.197 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 bench mirror/test2 --io-type write --io-size 4096 --io-threads 1 --io-total 409600 --io-pattern rand 2020-04-10T07:10:35.269 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:bench type write io_size 4096 io_threads 1 bytes 409600 pattern random 2020-04-10T07:10:35.339 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: SEC OPS OPS/SEC BYTES/SEC 2020-04-10T07:10:35.339 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:elapsed: 0 ops: 100 ops/sec: 1470.82 bytes/sec: 5.7 MiB/s 2020-04-10T07:10:35.350 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mirror_image_snapshot cluster2 mirror test2 2020-04-10T07:10:35.350 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:35.350 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:35.350 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test2 2020-04-10T07:10:35.350 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test2 2020-04-10T07:10:36.096 INFO:tasks.ceph.cluster2.osd.0.smithi204.stderr:2020-04-10T07:10:36.095+0000 7f034447a700 -1 reset not still connected to 0x55c29344b740 2020-04-10T07:10:36.135 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Snapshot ID: 49 2020-04-10T07:10:36.145 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_present cluster1 mirror test2 present 2020-04-10T07:10:36.146 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:36.146 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:36.146 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test2 2020-04-10T07:10:36.146 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state=present 2020-04-10T07:10:36.146 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image_id= 2020-04-10T07:10:36.146 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local s 2020-04-10T07:10:36.146 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 2020-04-10T07:10:36.147 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_image_id cluster1 mirror test2 2020-04-10T07:10:36.147 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:36.147 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:36.147 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test2 2020-04-10T07:10:36.147 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster1 info mirror/test2 2020-04-10T07:10:36.147 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -ne s/^.*block_name_prefix: rbd_data\.//p 2020-04-10T07:10:36.199 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ image_id=1143678674d2 2020-04-10T07:10:36.199 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0.1 2020-04-10T07:10:36.301 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_image_present cluster1 mirror test2 present 1143678674d2 2020-04-10T07:10:36.301 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:36.301 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:36.301 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test2 2020-04-10T07:10:36.302 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local test_state=present 2020-04-10T07:10:36.302 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image_id=1143678674d2 2020-04-10T07:10:36.302 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_state=deleted 2020-04-10T07:10:36.302 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_image_id 2020-04-10T07:10:36.302 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_image_id cluster1 mirror test2 2020-04-10T07:10:36.302 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:36.302 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:36.303 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test2 2020-04-10T07:10:36.303 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster1 info mirror/test2 2020-04-10T07:10:36.303 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -ne s/^.*block_name_prefix: rbd_data\.//p 2020-04-10T07:10:36.365 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ current_image_id=1143678674d2 2020-04-10T07:10:36.365 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 1143678674d2 2020-04-10T07:10:36.365 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -z 1143678674d2 -o 1143678674d2 = 1143678674d2 2020-04-10T07:10:36.365 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ current_state=present 2020-04-10T07:10:36.366 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test present = present 2020-04-10T07:10:36.366 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:10:36.366 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_snap_present cluster1 mirror test2 snap2 2020-04-10T07:10:36.366 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:36.366 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:36.366 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test2 2020-04-10T07:10:36.366 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snap_name=snap2 2020-04-10T07:10:36.367 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local s 2020-04-10T07:10:36.367 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:10:37.366 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 info mirror/test2@snap2 2020-04-10T07:10:37.425 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:error setting snapshot context: (2) No such file or directory 2020-04-10T07:10:37.431 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ continue 2020-04-10T07:10:37.431 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 2 2020-04-10T07:10:39.432 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 info mirror/test2@snap2 2020-04-10T07:10:39.487 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:rbd image 'test2': 2020-04-10T07:10:39.487 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: size 128 MiB in 32 objects 2020-04-10T07:10:39.487 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: order 22 (4 MiB objects) 2020-04-10T07:10:39.487 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: snapshot_count: 4 2020-04-10T07:10:39.487 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: id: 1143678674d2 2020-04-10T07:10:39.487 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: block_name_prefix: rbd_data.1143678674d2 2020-04-10T07:10:39.488 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: format: 2 2020-04-10T07:10:39.488 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: features: layering, exclusive-lock, journaling, non-primary 2020-04-10T07:10:39.488 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: op_features: 2020-04-10T07:10:39.488 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: flags: 2020-04-10T07:10:39.488 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: create_timestamp: Fri Apr 10 07:10:35 2020 2020-04-10T07:10:39.488 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: access_timestamp: Fri Apr 10 07:10:35 2020 2020-04-10T07:10:39.488 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: modify_timestamp: Fri Apr 10 07:10:35 2020 2020-04-10T07:10:39.489 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: protected: False 2020-04-10T07:10:39.489 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: journal: 1143678674d2 2020-04-10T07:10:39.489 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring state: enabled 2020-04-10T07:10:39.489 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring mode: snapshot 2020-04-10T07:10:39.489 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring global id: 925c4d02-9086-4edd-b4f3-79c1b7e3734c 2020-04-10T07:10:39.489 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring primary: false 2020-04-10T07:10:39.492 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:10:39.492 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ create_image_and_enable_mirror cluster2 mirror test3 2020-04-10T07:10:39.492 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:39.492 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:10:39.493 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:39.493 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:10:39.493 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test3 2020-04-10T07:10:39.493 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:10:39.493 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local mode=snapshot 2020-04-10T07:10:39.493 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:10:39.493 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ create_image cluster2 mirror test3 2020-04-10T07:10:39.494 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:39.494 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:10:39.494 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:39.494 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:10:39.494 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test3 2020-04-10T07:10:39.494 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:10:39.494 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local size=128 2020-04-10T07:10:39.495 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:10:39.495 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 create --size 128 --image-feature layering,exclusive-lock,journaling mirror/test3 2020-04-10T07:10:39.554 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ enable_mirror cluster2 mirror test3 snapshot 2020-04-10T07:10:39.555 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:39.555 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:39.555 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test3 2020-04-10T07:10:39.555 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local mode=snapshot 2020-04-10T07:10:39.555 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/test3 snapshot 2020-04-10T07:10:40.322 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Mirroring enabled 2020-04-10T07:10:40.331 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ write_image cluster2 mirror test3 100 2020-04-10T07:10:40.331 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:40.331 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:40.332 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test3 2020-04-10T07:10:40.332 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local count=100 2020-04-10T07:10:40.332 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local size= 2020-04-10T07:10:40.332 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 2020-04-10T07:10:40.332 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ size=4096 2020-04-10T07:10:40.332 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 bench mirror/test3 --io-type write --io-size 4096 --io-threads 1 --io-total 409600 --io-pattern rand 2020-04-10T07:10:40.408 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:bench type write io_size 4096 io_threads 1 bytes 409600 pattern random 2020-04-10T07:10:40.471 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: SEC OPS OPS/SEC BYTES/SEC 2020-04-10T07:10:40.471 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:elapsed: 0 ops: 100 ops/sec: 1562.71 bytes/sec: 6.1 MiB/s 2020-04-10T07:10:40.483 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ create_snapshot cluster2 mirror test3 snap1 2020-04-10T07:10:40.483 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:40.483 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:40.484 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test3 2020-04-10T07:10:40.484 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snap=snap1 2020-04-10T07:10:40.484 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 snap create mirror/test3@snap1 2020-04-10T07:10:41.316 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ create_snapshot cluster2 mirror test3 snap2 2020-04-10T07:10:41.316 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:41.317 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:41.317 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test3 2020-04-10T07:10:41.317 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snap=snap2 2020-04-10T07:10:41.317 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 snap create mirror/test3@snap2 2020-04-10T07:10:42.329 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ test3 = test4 ] 2020-04-10T07:10:42.329 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ test3 = test5 ] 2020-04-10T07:10:42.330 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ write_image cluster2 mirror test3 100 2020-04-10T07:10:42.330 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:42.330 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:42.330 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test3 2020-04-10T07:10:42.330 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local count=100 2020-04-10T07:10:42.330 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local size= 2020-04-10T07:10:42.331 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 2020-04-10T07:10:42.331 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ size=4096 2020-04-10T07:10:42.331 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 bench mirror/test3 --io-type write --io-size 4096 --io-threads 1 --io-total 409600 --io-pattern rand 2020-04-10T07:10:42.400 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:bench type write io_size 4096 io_threads 1 bytes 409600 pattern random 2020-04-10T07:10:42.466 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: SEC OPS OPS/SEC BYTES/SEC 2020-04-10T07:10:42.466 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:elapsed: 0 ops: 100 ops/sec: 1470.77 bytes/sec: 5.7 MiB/s 2020-04-10T07:10:42.480 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mirror_image_snapshot cluster2 mirror test3 2020-04-10T07:10:42.480 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:42.480 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:42.481 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test3 2020-04-10T07:10:42.481 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test3 2020-04-10T07:10:43.331 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Snapshot ID: 54 2020-04-10T07:10:43.341 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_present cluster1 mirror test3 present 2020-04-10T07:10:43.342 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:43.342 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:43.342 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test3 2020-04-10T07:10:43.342 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state=present 2020-04-10T07:10:43.342 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image_id= 2020-04-10T07:10:43.342 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local s 2020-04-10T07:10:43.343 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 2020-04-10T07:10:43.343 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_image_id cluster1 mirror test3 2020-04-10T07:10:43.343 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:43.343 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:43.343 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test3 2020-04-10T07:10:43.343 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster1 info mirror/test3 2020-04-10T07:10:43.343 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -ne s/^.*block_name_prefix: rbd_data\.//p 2020-04-10T07:10:43.400 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ image_id=114b4e325adf 2020-04-10T07:10:43.400 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0.1 2020-04-10T07:10:43.502 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_image_present cluster1 mirror test3 present 114b4e325adf 2020-04-10T07:10:43.503 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:43.503 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:43.503 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test3 2020-04-10T07:10:43.503 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local test_state=present 2020-04-10T07:10:43.503 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image_id=114b4e325adf 2020-04-10T07:10:43.503 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_state=deleted 2020-04-10T07:10:43.503 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_image_id 2020-04-10T07:10:43.504 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_image_id cluster1 mirror test3 2020-04-10T07:10:43.504 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:43.504 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:43.504 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test3 2020-04-10T07:10:43.504 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster1 info mirror/test3 2020-04-10T07:10:43.504 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -ne s/^.*block_name_prefix: rbd_data\.//p 2020-04-10T07:10:43.564 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ current_image_id=114b4e325adf 2020-04-10T07:10:43.565 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 114b4e325adf 2020-04-10T07:10:43.565 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -z 114b4e325adf -o 114b4e325adf = 114b4e325adf 2020-04-10T07:10:43.565 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ current_state=present 2020-04-10T07:10:43.565 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test present = present 2020-04-10T07:10:43.565 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:10:43.565 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_snap_present cluster1 mirror test3 snap2 2020-04-10T07:10:43.565 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:43.566 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:43.566 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test3 2020-04-10T07:10:43.566 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snap_name=snap2 2020-04-10T07:10:43.566 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local s 2020-04-10T07:10:43.566 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:10:44.565 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 info mirror/test3@snap2 2020-04-10T07:10:44.624 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:rbd image 'test3': 2020-04-10T07:10:44.624 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: size 128 MiB in 32 objects 2020-04-10T07:10:44.624 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: order 22 (4 MiB objects) 2020-04-10T07:10:44.624 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: snapshot_count: 3 2020-04-10T07:10:44.624 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: id: 114b4e325adf 2020-04-10T07:10:44.624 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: block_name_prefix: rbd_data.114b4e325adf 2020-04-10T07:10:44.625 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: format: 2 2020-04-10T07:10:44.625 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: features: layering, exclusive-lock, journaling, non-primary 2020-04-10T07:10:44.625 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: op_features: 2020-04-10T07:10:44.625 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: flags: 2020-04-10T07:10:44.625 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: create_timestamp: Fri Apr 10 07:10:42 2020 2020-04-10T07:10:44.625 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: access_timestamp: Fri Apr 10 07:10:42 2020 2020-04-10T07:10:44.625 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: modify_timestamp: Fri Apr 10 07:10:42 2020 2020-04-10T07:10:44.625 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: protected: False 2020-04-10T07:10:44.626 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: journal: 114b4e325adf 2020-04-10T07:10:44.626 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring state: enabled 2020-04-10T07:10:44.626 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring mode: snapshot 2020-04-10T07:10:44.626 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring global id: 3876de2e-b17d-4c86-94c4-90a11916f44e 2020-04-10T07:10:44.626 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring primary: false 2020-04-10T07:10:44.629 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:10:44.629 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ create_image_and_enable_mirror cluster2 mirror test4 2020-04-10T07:10:44.629 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:44.629 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:10:44.630 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:44.630 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:10:44.630 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test4 2020-04-10T07:10:44.630 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:10:44.630 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local mode=snapshot 2020-04-10T07:10:44.630 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:10:44.630 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ create_image cluster2 mirror test4 2020-04-10T07:10:44.631 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:44.631 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:10:44.631 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:44.631 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:10:44.631 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test4 2020-04-10T07:10:44.631 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:10:44.631 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local size=128 2020-04-10T07:10:44.631 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:10:44.632 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 create --size 128 --image-feature layering,exclusive-lock,journaling mirror/test4 2020-04-10T07:10:44.694 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ enable_mirror cluster2 mirror test4 snapshot 2020-04-10T07:10:44.694 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:44.695 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:44.695 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test4 2020-04-10T07:10:44.695 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local mode=snapshot 2020-04-10T07:10:44.695 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/test4 snapshot 2020-04-10T07:10:45.352 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Mirroring enabled 2020-04-10T07:10:45.361 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ write_image cluster2 mirror test4 100 2020-04-10T07:10:45.361 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:45.361 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:45.361 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test4 2020-04-10T07:10:45.361 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local count=100 2020-04-10T07:10:45.362 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local size= 2020-04-10T07:10:45.362 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 2020-04-10T07:10:45.362 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ size=4096 2020-04-10T07:10:45.362 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 bench mirror/test4 --io-type write --io-size 4096 --io-threads 1 --io-total 409600 --io-pattern rand 2020-04-10T07:10:45.430 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:bench type write io_size 4096 io_threads 1 bytes 409600 pattern random 2020-04-10T07:10:45.489 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: SEC OPS OPS/SEC BYTES/SEC 2020-04-10T07:10:45.490 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:elapsed: 0 ops: 100 ops/sec: 1785.92 bytes/sec: 7.0 MiB/s 2020-04-10T07:10:45.504 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ create_snapshot cluster2 mirror test4 snap1 2020-04-10T07:10:45.504 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:45.504 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:45.505 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test4 2020-04-10T07:10:45.505 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snap=snap1 2020-04-10T07:10:45.505 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 snap create mirror/test4@snap1 2020-04-10T07:10:46.158 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ create_snapshot cluster2 mirror test4 snap2 2020-04-10T07:10:46.158 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:46.158 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:46.158 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test4 2020-04-10T07:10:46.158 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snap=snap2 2020-04-10T07:10:46.158 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 snap create mirror/test4@snap2 2020-04-10T07:10:47.141 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ test4 = test4 ] 2020-04-10T07:10:47.141 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ protect_snapshot cluster2 mirror test4 snap1 2020-04-10T07:10:47.141 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:47.141 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:47.141 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test4 2020-04-10T07:10:47.141 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snap=snap1 2020-04-10T07:10:47.142 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 snap protect mirror/test4@snap1 2020-04-10T07:10:47.225 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ protect_snapshot cluster2 mirror test4 snap2 2020-04-10T07:10:47.225 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:47.225 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:47.225 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test4 2020-04-10T07:10:47.225 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snap=snap2 2020-04-10T07:10:47.225 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 snap protect mirror/test4@snap2 2020-04-10T07:10:47.314 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ write_image cluster2 mirror test4 100 2020-04-10T07:10:47.314 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:47.314 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:47.315 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test4 2020-04-10T07:10:47.315 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local count=100 2020-04-10T07:10:47.315 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local size= 2020-04-10T07:10:47.315 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 2020-04-10T07:10:47.316 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ size=4096 2020-04-10T07:10:47.316 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 bench mirror/test4 --io-type write --io-size 4096 --io-threads 1 --io-total 409600 --io-pattern rand 2020-04-10T07:10:47.389 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:bench type write io_size 4096 io_threads 1 bytes 409600 pattern random 2020-04-10T07:10:47.460 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: SEC OPS OPS/SEC BYTES/SEC 2020-04-10T07:10:47.460 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:elapsed: 0 ops: 100 ops/sec: 1389.04 bytes/sec: 5.4 MiB/s 2020-04-10T07:10:47.471 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mirror_image_snapshot cluster2 mirror test4 2020-04-10T07:10:47.471 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:47.471 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:47.471 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test4 2020-04-10T07:10:47.472 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test4 2020-04-10T07:10:48.365 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Snapshot ID: 59 2020-04-10T07:10:48.380 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_present cluster1 mirror test4 present 2020-04-10T07:10:48.380 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:48.380 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:48.380 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test4 2020-04-10T07:10:48.381 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state=present 2020-04-10T07:10:48.381 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image_id= 2020-04-10T07:10:48.381 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local s 2020-04-10T07:10:48.381 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 2020-04-10T07:10:48.381 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_image_id cluster1 mirror test4 2020-04-10T07:10:48.381 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:48.381 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:48.382 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test4 2020-04-10T07:10:48.382 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster1 info mirror/test4 2020-04-10T07:10:48.382 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -ne s/^.*block_name_prefix: rbd_data\.//p 2020-04-10T07:10:48.435 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ image_id=1151b1edf980 2020-04-10T07:10:48.436 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0.1 2020-04-10T07:10:48.538 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_image_present cluster1 mirror test4 present 1151b1edf980 2020-04-10T07:10:48.538 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:48.538 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:48.538 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test4 2020-04-10T07:10:48.538 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local test_state=present 2020-04-10T07:10:48.539 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image_id=1151b1edf980 2020-04-10T07:10:48.539 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_state=deleted 2020-04-10T07:10:48.539 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_image_id 2020-04-10T07:10:48.539 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_image_id cluster1 mirror test4 2020-04-10T07:10:48.539 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:48.539 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:48.539 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test4 2020-04-10T07:10:48.540 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster1 info mirror/test4 2020-04-10T07:10:48.540 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -ne s/^.*block_name_prefix: rbd_data\.//p 2020-04-10T07:10:48.597 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ current_image_id=1151b1edf980 2020-04-10T07:10:48.597 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 1151b1edf980 2020-04-10T07:10:48.598 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -z 1151b1edf980 -o 1151b1edf980 = 1151b1edf980 2020-04-10T07:10:48.598 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ current_state=present 2020-04-10T07:10:48.598 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test present = present 2020-04-10T07:10:48.598 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:10:48.598 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_snap_present cluster1 mirror test4 snap2 2020-04-10T07:10:48.598 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:48.598 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:48.599 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test4 2020-04-10T07:10:48.599 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snap_name=snap2 2020-04-10T07:10:48.599 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local s 2020-04-10T07:10:48.599 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:10:49.598 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 info mirror/test4@snap2 2020-04-10T07:10:49.653 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:rbd image 'test4': 2020-04-10T07:10:49.653 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: size 128 MiB in 32 objects 2020-04-10T07:10:49.653 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: order 22 (4 MiB objects) 2020-04-10T07:10:49.653 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: snapshot_count: 3 2020-04-10T07:10:49.653 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: id: 1151b1edf980 2020-04-10T07:10:49.654 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: block_name_prefix: rbd_data.1151b1edf980 2020-04-10T07:10:49.654 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: format: 2 2020-04-10T07:10:49.654 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: features: layering, exclusive-lock, journaling, non-primary 2020-04-10T07:10:49.654 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: op_features: 2020-04-10T07:10:49.654 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: flags: 2020-04-10T07:10:49.654 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: create_timestamp: Fri Apr 10 07:10:47 2020 2020-04-10T07:10:49.654 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: access_timestamp: Fri Apr 10 07:10:47 2020 2020-04-10T07:10:49.655 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: modify_timestamp: Fri Apr 10 07:10:47 2020 2020-04-10T07:10:49.655 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: protected: True 2020-04-10T07:10:49.655 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: journal: 1151b1edf980 2020-04-10T07:10:49.655 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring state: enabled 2020-04-10T07:10:49.655 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring mode: snapshot 2020-04-10T07:10:49.655 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring global id: 5d3e94e2-1c6c-4146-b461-67ecadf338c4 2020-04-10T07:10:49.655 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring primary: false 2020-04-10T07:10:49.658 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:10:49.658 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ create_image_and_enable_mirror cluster2 mirror test5 2020-04-10T07:10:49.658 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:49.658 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:10:49.659 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:49.659 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:10:49.659 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test5 2020-04-10T07:10:49.659 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:10:49.659 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local mode=snapshot 2020-04-10T07:10:49.659 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:10:49.659 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ create_image cluster2 mirror test5 2020-04-10T07:10:49.660 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:49.660 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:10:49.660 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:49.660 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:10:49.660 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test5 2020-04-10T07:10:49.660 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ shift 2020-04-10T07:10:49.660 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local size=128 2020-04-10T07:10:49.661 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n ] 2020-04-10T07:10:49.661 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 create --size 128 --image-feature layering,exclusive-lock,journaling mirror/test5 2020-04-10T07:10:49.723 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ enable_mirror cluster2 mirror test5 snapshot 2020-04-10T07:10:49.723 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:49.724 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:49.724 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test5 2020-04-10T07:10:49.724 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local mode=snapshot 2020-04-10T07:10:49.724 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/test5 snapshot 2020-04-10T07:10:50.378 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Mirroring enabled 2020-04-10T07:10:50.387 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ write_image cluster2 mirror test5 100 2020-04-10T07:10:50.388 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:50.388 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:50.388 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test5 2020-04-10T07:10:50.388 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local count=100 2020-04-10T07:10:50.389 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local size= 2020-04-10T07:10:50.389 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 2020-04-10T07:10:50.389 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ size=4096 2020-04-10T07:10:50.389 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 bench mirror/test5 --io-type write --io-size 4096 --io-threads 1 --io-total 409600 --io-pattern rand 2020-04-10T07:10:50.455 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:bench type write io_size 4096 io_threads 1 bytes 409600 pattern random 2020-04-10T07:10:50.522 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: SEC OPS OPS/SEC BYTES/SEC 2020-04-10T07:10:50.522 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:elapsed: 0 ops: 100 ops/sec: 1470.74 bytes/sec: 5.7 MiB/s 2020-04-10T07:10:50.537 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ create_snapshot cluster2 mirror test5 snap1 2020-04-10T07:10:50.538 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:50.538 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:50.538 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test5 2020-04-10T07:10:50.538 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snap=snap1 2020-04-10T07:10:50.538 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 snap create mirror/test5@snap1 2020-04-10T07:10:51.162 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ create_snapshot cluster2 mirror test5 snap2 2020-04-10T07:10:51.162 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:51.162 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:51.162 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test5 2020-04-10T07:10:51.162 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snap=snap2 2020-04-10T07:10:51.162 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 snap create mirror/test5@snap2 2020-04-10T07:10:52.144 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ test5 = test4 ] 2020-04-10T07:10:52.145 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ test5 = test5 ] 2020-04-10T07:10:52.146 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ protect_snapshot cluster2 mirror test5 snap1 2020-04-10T07:10:52.146 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:52.146 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:52.146 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test5 2020-04-10T07:10:52.146 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snap=snap1 2020-04-10T07:10:52.147 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 snap protect mirror/test5@snap1 2020-04-10T07:10:52.226 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ protect_snapshot cluster2 mirror test5 snap2 2020-04-10T07:10:52.226 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:52.226 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:52.226 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test5 2020-04-10T07:10:52.226 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snap=snap2 2020-04-10T07:10:52.227 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 snap protect mirror/test5@snap2 2020-04-10T07:10:52.309 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ write_image cluster2 mirror test5 100 2020-04-10T07:10:52.309 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:52.309 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:52.309 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test5 2020-04-10T07:10:52.310 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local count=100 2020-04-10T07:10:52.310 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local size= 2020-04-10T07:10:52.310 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 2020-04-10T07:10:52.310 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ size=4096 2020-04-10T07:10:52.310 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 bench mirror/test5 --io-type write --io-size 4096 --io-threads 1 --io-total 409600 --io-pattern rand 2020-04-10T07:10:52.390 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:bench type write io_size 4096 io_threads 1 bytes 409600 pattern random 2020-04-10T07:10:52.461 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: SEC OPS OPS/SEC BYTES/SEC 2020-04-10T07:10:52.461 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:elapsed: 0 ops: 100 ops/sec: 1389.02 bytes/sec: 5.4 MiB/s 2020-04-10T07:10:52.473 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mirror_image_snapshot cluster2 mirror test5 2020-04-10T07:10:52.473 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:52.473 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:52.473 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test5 2020-04-10T07:10:52.474 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test5 2020-04-10T07:10:53.384 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Snapshot ID: 64 2020-04-10T07:10:53.395 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_present cluster1 mirror test5 present 2020-04-10T07:10:53.395 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:53.395 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:53.396 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test5 2020-04-10T07:10:53.396 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state=present 2020-04-10T07:10:53.396 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image_id= 2020-04-10T07:10:53.396 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local s 2020-04-10T07:10:53.396 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 2020-04-10T07:10:53.397 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_image_id cluster1 mirror test5 2020-04-10T07:10:53.397 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:53.397 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:53.397 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test5 2020-04-10T07:10:53.397 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster1 info mirror/test5 2020-04-10T07:10:53.397 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -ne s/^.*block_name_prefix: rbd_data\.//p 2020-04-10T07:10:53.453 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ image_id=113fb52188b7 2020-04-10T07:10:53.453 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0.1 2020-04-10T07:10:53.555 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_image_present cluster1 mirror test5 present 113fb52188b7 2020-04-10T07:10:53.556 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:53.556 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:53.556 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test5 2020-04-10T07:10:53.556 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local test_state=present 2020-04-10T07:10:53.556 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image_id=113fb52188b7 2020-04-10T07:10:53.556 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_state=deleted 2020-04-10T07:10:53.557 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_image_id 2020-04-10T07:10:53.557 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_image_id cluster1 mirror test5 2020-04-10T07:10:53.557 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:53.557 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:53.557 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test5 2020-04-10T07:10:53.557 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster1 info mirror/test5 2020-04-10T07:10:53.557 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -ne s/^.*block_name_prefix: rbd_data\.//p 2020-04-10T07:10:53.618 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ current_image_id=113fb52188b7 2020-04-10T07:10:53.618 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 113fb52188b7 2020-04-10T07:10:53.619 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -z 113fb52188b7 -o 113fb52188b7 = 113fb52188b7 2020-04-10T07:10:53.619 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ current_state=present 2020-04-10T07:10:53.619 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test present = present 2020-04-10T07:10:53.619 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:10:53.619 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_snap_present cluster1 mirror test5 snap2 2020-04-10T07:10:53.619 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:10:53.620 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:53.620 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test5 2020-04-10T07:10:53.620 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snap_name=snap2 2020-04-10T07:10:53.620 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local s 2020-04-10T07:10:53.620 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:10:54.619 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 info mirror/test5@snap2 2020-04-10T07:10:54.669 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:rbd image 'test5': 2020-04-10T07:10:54.670 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: size 128 MiB in 32 objects 2020-04-10T07:10:54.670 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: order 22 (4 MiB objects) 2020-04-10T07:10:54.670 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: snapshot_count: 3 2020-04-10T07:10:54.670 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: id: 113fb52188b7 2020-04-10T07:10:54.670 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: block_name_prefix: rbd_data.113fb52188b7 2020-04-10T07:10:54.671 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: format: 2 2020-04-10T07:10:54.671 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: features: layering, exclusive-lock, journaling, non-primary 2020-04-10T07:10:54.671 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: op_features: 2020-04-10T07:10:54.671 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: flags: 2020-04-10T07:10:54.671 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: create_timestamp: Fri Apr 10 07:10:52 2020 2020-04-10T07:10:54.672 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: access_timestamp: Fri Apr 10 07:10:52 2020 2020-04-10T07:10:54.672 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: modify_timestamp: Fri Apr 10 07:10:52 2020 2020-04-10T07:10:54.672 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: protected: True 2020-04-10T07:10:54.672 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: journal: 113fb52188b7 2020-04-10T07:10:54.672 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring state: enabled 2020-04-10T07:10:54.672 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring mode: snapshot 2020-04-10T07:10:54.673 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring global id: 697ea219-b252-4c33-ba20-9b31ea5be82a 2020-04-10T07:10:54.673 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring primary: false 2020-04-10T07:10:54.674 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:10:54.674 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_pool_mirror_mode cluster2 mirror image 2020-04-10T07:10:54.674 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:54.675 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:54.675 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local mode=image 2020-04-10T07:10:54.675 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster2 mirror pool enable mirror image 2020-04-10T07:10:54.712 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:rbd: mirroring is already configured for image mode 2020-04-10T07:10:54.716 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ disable_mirror cluster2 mirror test2 2020-04-10T07:10:54.716 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:54.716 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:54.716 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test2 2020-04-10T07:10:54.716 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster2 mirror image disable mirror/test2 2020-04-10T07:10:55.404 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Mirroring disabled 2020-04-10T07:10:55.409 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ disable_mirror cluster2 mirror test4 2020-04-10T07:10:55.409 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:55.410 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:55.410 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test4 2020-04-10T07:10:55.410 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster2 mirror image disable mirror/test4 2020-04-10T07:10:56.190 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Mirroring disabled 2020-04-10T07:10:56.197 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ unprotect_snapshot cluster2 mirror test5 snap1 2020-04-10T07:10:56.197 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:56.197 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:56.197 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test5 2020-04-10T07:10:56.197 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snap=snap1 2020-04-10T07:10:56.198 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 snap unprotect mirror/test5@snap1 2020-04-10T07:10:56.261 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ unprotect_snapshot cluster2 mirror test5 snap2 2020-04-10T07:10:56.261 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:56.261 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:56.261 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test5 2020-04-10T07:10:56.262 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snap=snap2 2020-04-10T07:10:56.262 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 snap unprotect mirror/test5@snap2 2020-04-10T07:10:56.329 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ remove_snapshot cluster2 mirror test3 snap1 2020-04-10T07:10:56.329 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:56.329 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:56.329 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test3 2020-04-10T07:10:56.330 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snap=snap1 2020-04-10T07:10:56.330 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 snap rm mirror/test3@snap1 2020-04-10T07:10:57.412 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: Removing snap: 100% complete...done. 2020-04-10T07:10:57.421 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ remove_snapshot cluster2 mirror test3 snap2 2020-04-10T07:10:57.421 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:57.421 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:57.421 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test3 2020-04-10T07:10:57.421 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snap=snap2 2020-04-10T07:10:57.422 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 snap rm mirror/test3@snap2 2020-04-10T07:10:58.423 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: Removing snap: 100% complete...done. 2020-04-10T07:10:58.431 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ remove_image_retry cluster2 mirror test3 2020-04-10T07:10:58.432 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:58.432 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:58.432 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test3 2020-04-10T07:10:58.432 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0 2020-04-10T07:10:58.433 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ remove_image cluster2 mirror test3 2020-04-10T07:10:58.433 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:58.433 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:58.433 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test3 2020-04-10T07:10:58.433 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster2 snap purge mirror/test3 2020-04-10T07:10:58.479 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster2 rm mirror/test3 2020-04-10T07:10:58.625 INFO:tasks.ceph.cluster1.osd.0.smithi101.stderr:2020-04-10T07:10:58.620+0000 7fbf46058700 -1 osd.0 pg_epoch: 99 pg[3.2c( v 99'24 (0'0,99'24] local-lis/les=14/15 n=12 ec=14/14 lis/c=14/14 les/c/f=15/15/0 sis=14) [0,1] r=0 lpr=14 luod=98'23 crt=99'24 lcod 97'21 mlcod 97'21 active+clean+snaptrim trimq=[30~1] ps=[2d~3]] removing snap head 2020-04-10T07:10:58.629 INFO:tasks.ceph.cluster1.osd.0.smithi101.stderr:2020-04-10T07:10:58.624+0000 7fbf4084d700 -1 osd.0 pg_epoch: 99 pg[3.1f( v 99'75 (0'0,99'75] local-lis/les=14/15 n=7 ec=14/14 lis/c=14/14 les/c/f=15/15/0 sis=14) [0,1] r=0 lpr=14 crt=99'75 lcod 98'74 mlcod 98'74 active+clean+snaptrim trimq=[30~1] ps=[2d~3]] removing snap head 2020-04-10T07:10:59.506 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2020-04-10T07:10:59.509 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:10:59.509 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ remove_snapshot cluster2 mirror test5 snap1 2020-04-10T07:10:59.509 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:10:59.509 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:10:59.510 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test5 2020-04-10T07:10:59.510 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snap=snap1 2020-04-10T07:10:59.510 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 snap rm mirror/test5@snap1 2020-04-10T07:11:00.451 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: Removing snap: 100% complete...done. 2020-04-10T07:11:00.456 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ remove_snapshot cluster2 mirror test5 snap2 2020-04-10T07:11:00.456 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:11:00.456 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:11:00.457 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test5 2020-04-10T07:11:00.457 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snap=snap2 2020-04-10T07:11:00.457 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 snap rm mirror/test5@snap2 2020-04-10T07:11:01.166 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: Removing snap: 100% complete...done. 2020-04-10T07:11:01.177 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ remove_image_retry cluster2 mirror test5 2020-04-10T07:11:01.177 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:11:01.177 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:11:01.177 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test5 2020-04-10T07:11:01.178 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0 2020-04-10T07:11:01.178 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ remove_image cluster2 mirror test5 2020-04-10T07:11:01.178 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:11:01.178 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:11:01.179 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test5 2020-04-10T07:11:01.179 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster2 snap purge mirror/test5 2020-04-10T07:11:01.224 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster2 rm mirror/test5 2020-04-10T07:11:02.236 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2020-04-10T07:11:02.240 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:11:02.240 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_present cluster1 mirror test2 deleted 2020-04-10T07:11:02.240 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:11:02.240 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:11:02.240 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test2 2020-04-10T07:11:02.240 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state=deleted 2020-04-10T07:11:02.241 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image_id= 2020-04-10T07:11:02.241 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local s 2020-04-10T07:11:02.241 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 2020-04-10T07:11:02.241 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_image_id cluster1 mirror test2 2020-04-10T07:11:02.241 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:11:02.241 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:11:02.241 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test2 2020-04-10T07:11:02.242 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster1 info mirror/test2 2020-04-10T07:11:02.242 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -ne s/^.*block_name_prefix: rbd_data\.//p 2020-04-10T07:11:02.268 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: error opening image test2: (2) No such file or directory 2020-04-10T07:11:02.271 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ image_id= 2020-04-10T07:11:02.271 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0.1 2020-04-10T07:11:02.373 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_image_present cluster1 mirror test2 deleted 2020-04-10T07:11:02.373 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:11:02.374 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:11:02.374 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test2 2020-04-10T07:11:02.374 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local test_state=deleted 2020-04-10T07:11:02.374 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image_id= 2020-04-10T07:11:02.374 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_state=deleted 2020-04-10T07:11:02.374 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_image_id 2020-04-10T07:11:02.374 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_image_id cluster1 mirror test2 2020-04-10T07:11:02.375 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:11:02.375 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:11:02.375 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test2 2020-04-10T07:11:02.375 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster1 info mirror/test2 2020-04-10T07:11:02.375 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -ne s/^.*block_name_prefix: rbd_data\.//p 2020-04-10T07:11:02.412 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: error opening image test2: (2) No such file or directory 2020-04-10T07:11:02.417 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ current_image_id= 2020-04-10T07:11:02.417 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 2020-04-10T07:11:02.417 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test deleted = deleted 2020-04-10T07:11:02.418 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:11:02.418 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_present cluster1 mirror test3 deleted 2020-04-10T07:11:02.418 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:11:02.418 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:11:02.418 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test3 2020-04-10T07:11:02.418 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state=deleted 2020-04-10T07:11:02.418 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image_id= 2020-04-10T07:11:02.419 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local s 2020-04-10T07:11:02.419 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 2020-04-10T07:11:02.419 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_image_id cluster1 mirror test3 2020-04-10T07:11:02.419 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:11:02.419 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:11:02.419 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test3 2020-04-10T07:11:02.419 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster1 info mirror/test3 2020-04-10T07:11:02.419 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -ne s/^.*block_name_prefix: rbd_data\.//p 2020-04-10T07:11:02.490 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ image_id=114b4e325adf 2020-04-10T07:11:02.490 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0.1 2020-04-10T07:11:02.592 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_image_present cluster1 mirror test3 deleted 114b4e325adf 2020-04-10T07:11:02.592 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:11:02.592 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:11:02.592 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test3 2020-04-10T07:11:02.593 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local test_state=deleted 2020-04-10T07:11:02.593 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image_id=114b4e325adf 2020-04-10T07:11:02.593 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_state=deleted 2020-04-10T07:11:02.593 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_image_id 2020-04-10T07:11:02.593 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_image_id cluster1 mirror test3 2020-04-10T07:11:02.593 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:11:02.593 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:11:02.594 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test3 2020-04-10T07:11:02.594 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster1 info mirror/test3 2020-04-10T07:11:02.594 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -ne s/^.*block_name_prefix: rbd_data\.//p 2020-04-10T07:11:02.660 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ current_image_id=114b4e325adf 2020-04-10T07:11:02.660 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 114b4e325adf 2020-04-10T07:11:02.661 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -z 114b4e325adf -o 114b4e325adf = 114b4e325adf 2020-04-10T07:11:02.661 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ current_state=present 2020-04-10T07:11:02.661 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test deleted = present 2020-04-10T07:11:02.661 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:11:03.662 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_image_present cluster1 mirror test3 deleted 114b4e325adf 2020-04-10T07:11:03.662 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:11:03.662 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:11:03.662 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test3 2020-04-10T07:11:03.663 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local test_state=deleted 2020-04-10T07:11:03.663 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image_id=114b4e325adf 2020-04-10T07:11:03.663 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_state=deleted 2020-04-10T07:11:03.663 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_image_id 2020-04-10T07:11:03.663 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_image_id cluster1 mirror test3 2020-04-10T07:11:03.663 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:11:03.663 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:11:03.664 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test3 2020-04-10T07:11:03.664 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster1 info mirror/test3 2020-04-10T07:11:03.664 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -ne s/^.*block_name_prefix: rbd_data\.//p 2020-04-10T07:11:03.791 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ current_image_id=114b4e325adf 2020-04-10T07:11:03.791 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 114b4e325adf 2020-04-10T07:11:03.792 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -z 114b4e325adf -o 114b4e325adf = 114b4e325adf 2020-04-10T07:11:03.792 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ current_state=present 2020-04-10T07:11:03.792 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test deleted = present 2020-04-10T07:11:03.792 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 2 2020-04-10T07:11:05.796 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_image_present cluster1 mirror test3 deleted 114b4e325adf 2020-04-10T07:11:05.796 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:11:05.796 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:11:05.796 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test3 2020-04-10T07:11:05.796 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local test_state=deleted 2020-04-10T07:11:05.797 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image_id=114b4e325adf 2020-04-10T07:11:05.797 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_state=deleted 2020-04-10T07:11:05.797 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_image_id 2020-04-10T07:11:05.797 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_image_id cluster1 mirror test3 2020-04-10T07:11:05.797 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:11:05.797 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:11:05.797 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test3 2020-04-10T07:11:05.798 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster1 info mirror/test3 2020-04-10T07:11:05.798 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -ne s/^.*block_name_prefix: rbd_data\.//p 2020-04-10T07:11:05.834 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: error opening image test3: (2) No such file or directory 2020-04-10T07:11:05.838 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ current_image_id= 2020-04-10T07:11:05.839 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 2020-04-10T07:11:05.839 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test deleted = deleted 2020-04-10T07:11:05.839 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:11:05.839 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_present cluster1 mirror test4 deleted 2020-04-10T07:11:05.839 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:11:05.839 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:11:05.839 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test4 2020-04-10T07:11:05.840 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state=deleted 2020-04-10T07:11:05.840 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image_id= 2020-04-10T07:11:05.840 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local s 2020-04-10T07:11:05.840 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 2020-04-10T07:11:05.840 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_image_id cluster1 mirror test4 2020-04-10T07:11:05.840 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:11:05.840 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:11:05.841 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test4 2020-04-10T07:11:05.841 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster1 info mirror/test4 2020-04-10T07:11:05.842 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -ne s/^.*block_name_prefix: rbd_data\.//p 2020-04-10T07:11:05.877 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: error opening image test4: (2) No such file or directory 2020-04-10T07:11:05.880 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ image_id= 2020-04-10T07:11:05.881 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0.1 2020-04-10T07:11:05.982 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_image_present cluster1 mirror test4 deleted 2020-04-10T07:11:05.983 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:11:05.983 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:11:05.983 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test4 2020-04-10T07:11:05.983 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local test_state=deleted 2020-04-10T07:11:05.983 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image_id= 2020-04-10T07:11:05.983 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_state=deleted 2020-04-10T07:11:05.983 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_image_id 2020-04-10T07:11:05.984 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_image_id cluster1 mirror test4 2020-04-10T07:11:05.984 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:11:05.984 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:11:05.984 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test4 2020-04-10T07:11:05.984 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster1 info mirror/test4 2020-04-10T07:11:05.984 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -ne s/^.*block_name_prefix: rbd_data\.//p 2020-04-10T07:11:06.021 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: error opening image test4: (2) No such file or directory 2020-04-10T07:11:06.027 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ current_image_id= 2020-04-10T07:11:06.027 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 2020-04-10T07:11:06.027 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test deleted = deleted 2020-04-10T07:11:06.027 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:11:06.028 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_image_present cluster1 mirror test5 deleted 2020-04-10T07:11:06.028 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:11:06.028 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:11:06.028 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test5 2020-04-10T07:11:06.028 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local state=deleted 2020-04-10T07:11:06.028 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image_id= 2020-04-10T07:11:06.028 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local s 2020-04-10T07:11:06.028 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 2020-04-10T07:11:06.029 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_image_id cluster1 mirror test5 2020-04-10T07:11:06.029 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:11:06.029 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:11:06.029 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test5 2020-04-10T07:11:06.029 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster1 info mirror/test5 2020-04-10T07:11:06.029 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -ne s/^.*block_name_prefix: rbd_data\.//p 2020-04-10T07:11:06.067 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: error opening image test5: (2) No such file or directory 2020-04-10T07:11:06.069 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ image_id= 2020-04-10T07:11:06.069 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 0.1 2020-04-10T07:11:06.170 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test_image_present cluster1 mirror test5 deleted 2020-04-10T07:11:06.170 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:11:06.171 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:11:06.171 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test5 2020-04-10T07:11:06.171 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local test_state=deleted 2020-04-10T07:11:06.171 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image_id= 2020-04-10T07:11:06.171 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_state=deleted 2020-04-10T07:11:06.171 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local current_image_id 2020-04-10T07:11:06.171 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ get_image_id cluster1 mirror test5 2020-04-10T07:11:06.171 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:11:06.172 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:11:06.172 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test5 2020-04-10T07:11:06.172 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster1 info mirror/test5 2020-04-10T07:11:06.172 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sed -ne s/^.*block_name_prefix: rbd_data\.//p 2020-04-10T07:11:06.213 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: error opening image test5: (2) No such file or directory 2020-04-10T07:11:06.216 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ current_image_id= 2020-04-10T07:11:06.216 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 2020-04-10T07:11:06.216 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test deleted = deleted 2020-04-10T07:11:06.216 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:11:06.217 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ testlog TEST: snapshot rename 2020-04-10T07:11:06.217 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ tee -a /tmp/tmp.8SwBCsYPSC/rbd-mirror.test.log 2020-04-10T07:11:06.217 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ date +%F %T 2020-04-10T07:11:06.218 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10 07:11:06 TEST: snapshot rename 2020-04-10T07:11:06.218 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:2020-04-10 07:11:06 TEST: snapshot rename 2020-04-10T07:11:06.218 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ snap_name=snap_rename 2020-04-10T07:11:06.218 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ enable_mirror cluster2 mirror test2 2020-04-10T07:11:06.219 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:11:06.219 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:11:06.219 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test2 2020-04-10T07:11:06.219 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local mode=snapshot 2020-04-10T07:11:06.219 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/test2 snapshot 2020-04-10T07:11:07.163 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Mirroring enabled 2020-04-10T07:11:07.174 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ create_snapshot cluster2 mirror test2 snap_rename_0 2020-04-10T07:11:07.175 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:11:07.175 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:11:07.175 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test2 2020-04-10T07:11:07.175 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snap=snap_rename_0 2020-04-10T07:11:07.175 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 snap create mirror/test2@snap_rename_0 2020-04-10T07:11:08.159 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 1 20 2020-04-10T07:11:08.160 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ expr 1 - 1 2020-04-10T07:11:08.161 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rename_snapshot cluster2 mirror test2 snap_rename_0 snap_rename_1 2020-04-10T07:11:08.161 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:11:08.161 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:11:08.161 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test2 2020-04-10T07:11:08.162 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snap=snap_rename_0 2020-04-10T07:11:08.162 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local new_snap=snap_rename_1 2020-04-10T07:11:08.162 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 snap rename mirror/test2@snap_rename_0 mirror/test2@snap_rename_1 2020-04-10T07:11:08.247 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ expr 2 - 1 2020-04-10T07:11:08.248 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rename_snapshot cluster2 mirror test2 snap_rename_1 snap_rename_2 2020-04-10T07:11:08.248 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:11:08.248 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:11:08.248 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test2 2020-04-10T07:11:08.249 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snap=snap_rename_1 2020-04-10T07:11:08.249 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local new_snap=snap_rename_2 2020-04-10T07:11:08.249 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 snap rename mirror/test2@snap_rename_1 mirror/test2@snap_rename_2 2020-04-10T07:11:08.337 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ expr 3 - 1 2020-04-10T07:11:08.338 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rename_snapshot cluster2 mirror test2 snap_rename_2 snap_rename_3 2020-04-10T07:11:08.338 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:11:08.338 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:11:08.339 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test2 2020-04-10T07:11:08.339 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snap=snap_rename_2 2020-04-10T07:11:08.339 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local new_snap=snap_rename_3 2020-04-10T07:11:08.339 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 snap rename mirror/test2@snap_rename_2 mirror/test2@snap_rename_3 2020-04-10T07:11:08.634 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ expr 4 - 1 2020-04-10T07:11:08.634 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rename_snapshot cluster2 mirror test2 snap_rename_3 snap_rename_4 2020-04-10T07:11:08.635 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:11:08.635 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:11:08.635 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test2 2020-04-10T07:11:08.635 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snap=snap_rename_3 2020-04-10T07:11:08.635 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local new_snap=snap_rename_4 2020-04-10T07:11:08.635 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 snap rename mirror/test2@snap_rename_3 mirror/test2@snap_rename_4 2020-04-10T07:11:08.724 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ expr 5 - 1 2020-04-10T07:11:08.724 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rename_snapshot cluster2 mirror test2 snap_rename_4 snap_rename_5 2020-04-10T07:11:08.725 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:11:08.725 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:11:08.725 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test2 2020-04-10T07:11:08.725 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snap=snap_rename_4 2020-04-10T07:11:08.725 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local new_snap=snap_rename_5 2020-04-10T07:11:08.725 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 snap rename mirror/test2@snap_rename_4 mirror/test2@snap_rename_5 2020-04-10T07:11:08.816 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ expr 6 - 1 2020-04-10T07:11:08.817 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rename_snapshot cluster2 mirror test2 snap_rename_5 snap_rename_6 2020-04-10T07:11:08.817 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:11:08.817 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:11:08.817 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test2 2020-04-10T07:11:08.817 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snap=snap_rename_5 2020-04-10T07:11:08.818 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local new_snap=snap_rename_6 2020-04-10T07:11:08.818 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 snap rename mirror/test2@snap_rename_5 mirror/test2@snap_rename_6 2020-04-10T07:11:08.903 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ expr 7 - 1 2020-04-10T07:11:08.903 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rename_snapshot cluster2 mirror test2 snap_rename_6 snap_rename_7 2020-04-10T07:11:08.903 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:11:08.904 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:11:08.904 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test2 2020-04-10T07:11:08.904 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snap=snap_rename_6 2020-04-10T07:11:08.904 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local new_snap=snap_rename_7 2020-04-10T07:11:08.904 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 snap rename mirror/test2@snap_rename_6 mirror/test2@snap_rename_7 2020-04-10T07:11:08.985 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ expr 8 - 1 2020-04-10T07:11:08.986 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rename_snapshot cluster2 mirror test2 snap_rename_7 snap_rename_8 2020-04-10T07:11:08.986 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:11:08.986 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:11:08.986 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test2 2020-04-10T07:11:08.986 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snap=snap_rename_7 2020-04-10T07:11:08.986 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local new_snap=snap_rename_8 2020-04-10T07:11:08.986 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 snap rename mirror/test2@snap_rename_7 mirror/test2@snap_rename_8 2020-04-10T07:11:09.072 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ expr 9 - 1 2020-04-10T07:11:09.073 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rename_snapshot cluster2 mirror test2 snap_rename_8 snap_rename_9 2020-04-10T07:11:09.073 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:11:09.073 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:11:09.073 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test2 2020-04-10T07:11:09.073 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snap=snap_rename_8 2020-04-10T07:11:09.073 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local new_snap=snap_rename_9 2020-04-10T07:11:09.073 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 snap rename mirror/test2@snap_rename_8 mirror/test2@snap_rename_9 2020-04-10T07:11:09.157 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ expr 10 - 1 2020-04-10T07:11:09.158 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rename_snapshot cluster2 mirror test2 snap_rename_9 snap_rename_10 2020-04-10T07:11:09.158 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:11:09.158 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:11:09.158 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test2 2020-04-10T07:11:09.158 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snap=snap_rename_9 2020-04-10T07:11:09.158 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local new_snap=snap_rename_10 2020-04-10T07:11:09.159 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 snap rename mirror/test2@snap_rename_9 mirror/test2@snap_rename_10 2020-04-10T07:11:09.243 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ expr 11 - 1 2020-04-10T07:11:09.244 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rename_snapshot cluster2 mirror test2 snap_rename_10 snap_rename_11 2020-04-10T07:11:09.244 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:11:09.244 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:11:09.245 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test2 2020-04-10T07:11:09.245 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snap=snap_rename_10 2020-04-10T07:11:09.245 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local new_snap=snap_rename_11 2020-04-10T07:11:09.245 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 snap rename mirror/test2@snap_rename_10 mirror/test2@snap_rename_11 2020-04-10T07:11:09.337 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ expr 12 - 1 2020-04-10T07:11:09.337 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rename_snapshot cluster2 mirror test2 snap_rename_11 snap_rename_12 2020-04-10T07:11:09.337 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:11:09.338 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:11:09.338 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test2 2020-04-10T07:11:09.338 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snap=snap_rename_11 2020-04-10T07:11:09.338 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local new_snap=snap_rename_12 2020-04-10T07:11:09.338 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 snap rename mirror/test2@snap_rename_11 mirror/test2@snap_rename_12 2020-04-10T07:11:09.420 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ expr 13 - 1 2020-04-10T07:11:09.421 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rename_snapshot cluster2 mirror test2 snap_rename_12 snap_rename_13 2020-04-10T07:11:09.421 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:11:09.421 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:11:09.421 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test2 2020-04-10T07:11:09.422 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snap=snap_rename_12 2020-04-10T07:11:09.422 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local new_snap=snap_rename_13 2020-04-10T07:11:09.422 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 snap rename mirror/test2@snap_rename_12 mirror/test2@snap_rename_13 2020-04-10T07:11:09.507 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ expr 14 - 1 2020-04-10T07:11:09.507 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rename_snapshot cluster2 mirror test2 snap_rename_13 snap_rename_14 2020-04-10T07:11:09.507 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:11:09.507 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:11:09.508 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test2 2020-04-10T07:11:09.508 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snap=snap_rename_13 2020-04-10T07:11:09.508 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local new_snap=snap_rename_14 2020-04-10T07:11:09.508 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 snap rename mirror/test2@snap_rename_13 mirror/test2@snap_rename_14 2020-04-10T07:11:09.586 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ expr 15 - 1 2020-04-10T07:11:09.587 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rename_snapshot cluster2 mirror test2 snap_rename_14 snap_rename_15 2020-04-10T07:11:09.587 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:11:09.587 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:11:09.587 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test2 2020-04-10T07:11:09.587 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snap=snap_rename_14 2020-04-10T07:11:09.587 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local new_snap=snap_rename_15 2020-04-10T07:11:09.588 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 snap rename mirror/test2@snap_rename_14 mirror/test2@snap_rename_15 2020-04-10T07:11:09.680 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ expr 16 - 1 2020-04-10T07:11:09.680 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rename_snapshot cluster2 mirror test2 snap_rename_15 snap_rename_16 2020-04-10T07:11:09.680 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:11:09.681 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:11:09.681 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test2 2020-04-10T07:11:09.681 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snap=snap_rename_15 2020-04-10T07:11:09.681 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local new_snap=snap_rename_16 2020-04-10T07:11:09.681 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 snap rename mirror/test2@snap_rename_15 mirror/test2@snap_rename_16 2020-04-10T07:11:09.763 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ expr 17 - 1 2020-04-10T07:11:09.764 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rename_snapshot cluster2 mirror test2 snap_rename_16 snap_rename_17 2020-04-10T07:11:09.765 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:11:09.765 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:11:09.765 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test2 2020-04-10T07:11:09.765 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snap=snap_rename_16 2020-04-10T07:11:09.765 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local new_snap=snap_rename_17 2020-04-10T07:11:09.765 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 snap rename mirror/test2@snap_rename_16 mirror/test2@snap_rename_17 2020-04-10T07:11:09.849 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ expr 18 - 1 2020-04-10T07:11:09.850 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rename_snapshot cluster2 mirror test2 snap_rename_17 snap_rename_18 2020-04-10T07:11:09.850 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:11:09.850 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:11:09.851 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test2 2020-04-10T07:11:09.851 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snap=snap_rename_17 2020-04-10T07:11:09.851 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local new_snap=snap_rename_18 2020-04-10T07:11:09.851 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 snap rename mirror/test2@snap_rename_17 mirror/test2@snap_rename_18 2020-04-10T07:11:09.941 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ expr 19 - 1 2020-04-10T07:11:09.942 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rename_snapshot cluster2 mirror test2 snap_rename_18 snap_rename_19 2020-04-10T07:11:09.942 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:11:09.942 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:11:09.942 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test2 2020-04-10T07:11:09.943 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snap=snap_rename_18 2020-04-10T07:11:09.943 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local new_snap=snap_rename_19 2020-04-10T07:11:09.943 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 snap rename mirror/test2@snap_rename_18 mirror/test2@snap_rename_19 2020-04-10T07:11:10.029 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ expr 20 - 1 2020-04-10T07:11:10.030 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rename_snapshot cluster2 mirror test2 snap_rename_19 snap_rename_20 2020-04-10T07:11:10.030 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:11:10.031 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:11:10.031 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test2 2020-04-10T07:11:10.031 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snap=snap_rename_19 2020-04-10T07:11:10.031 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local new_snap=snap_rename_20 2020-04-10T07:11:10.031 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 snap rename mirror/test2@snap_rename_19 mirror/test2@snap_rename_20 2020-04-10T07:11:10.119 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ mirror_image_snapshot cluster2 mirror test2 2020-04-10T07:11:10.119 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:11:10.119 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:11:10.120 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test2 2020-04-10T07:11:10.120 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test2 2020-04-10T07:11:11.161 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Snapshot ID: 76 2020-04-10T07:11:11.171 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ wait_for_snap_present cluster1 mirror test2 snap_rename_20 2020-04-10T07:11:11.171 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:11:11.171 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pool=mirror 2020-04-10T07:11:11.171 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local image=test2 2020-04-10T07:11:11.171 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local snap_name=snap_rename_20 2020-04-10T07:11:11.172 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local s 2020-04-10T07:11:11.172 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:11:12.172 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 info mirror/test2@snap_rename_20 2020-04-10T07:11:12.223 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:error setting snapshot context: (2) No such file or directory 2020-04-10T07:11:12.226 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ continue 2020-04-10T07:11:12.226 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 2 2020-04-10T07:11:14.230 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 info mirror/test2@snap_rename_20 2020-04-10T07:11:14.291 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:rbd image 'test2': 2020-04-10T07:11:14.291 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: size 128 MiB in 32 objects 2020-04-10T07:11:14.291 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: order 22 (4 MiB objects) 2020-04-10T07:11:14.291 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: snapshot_count: 5 2020-04-10T07:11:14.291 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: id: 11439010b41 2020-04-10T07:11:14.292 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: block_name_prefix: rbd_data.11439010b41 2020-04-10T07:11:14.292 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: format: 2 2020-04-10T07:11:14.292 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: features: layering, exclusive-lock, journaling, non-primary 2020-04-10T07:11:14.292 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: op_features: 2020-04-10T07:11:14.292 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: flags: 2020-04-10T07:11:14.292 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: create_timestamp: Fri Apr 10 07:11:09 2020 2020-04-10T07:11:14.292 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: access_timestamp: Fri Apr 10 07:11:09 2020 2020-04-10T07:11:14.293 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: modify_timestamp: Fri Apr 10 07:11:09 2020 2020-04-10T07:11:14.293 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: protected: False 2020-04-10T07:11:14.293 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: journal: 11439010b41 2020-04-10T07:11:14.293 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring state: enabled 2020-04-10T07:11:14.293 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring mode: snapshot 2020-04-10T07:11:14.293 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring global id: b583e294-8b69-4e57-9966-783595c454d6 2020-04-10T07:11:14.293 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring primary: false 2020-04-10T07:11:14.305 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ return 0 2020-04-10T07:11:14.305 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ testlog TEST: disable mirror while daemon is stopped 2020-04-10T07:11:14.306 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ tee -a /tmp/tmp.8SwBCsYPSC/rbd-mirror.test.log 2020-04-10T07:11:14.306 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ date +%F %T 2020-04-10T07:11:14.306 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10 07:11:14 TEST: disable mirror while daemon is stopped 2020-04-10T07:11:14.306 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:2020-04-10 07:11:14 TEST: disable mirror while daemon is stopped 2020-04-10T07:11:14.306 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ stop_mirrors cluster1 2020-04-10T07:11:14.306 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:11:14.306 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local sig= 2020-04-10T07:11:14.307 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:11:14.307 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ stop_mirror cluster1:0 2020-04-10T07:11:14.307 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:11:14.308 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local sig= 2020-04-10T07:11:14.308 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 2020-04-10T07:11:14.308 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pid 2020-04-10T07:11:14.308 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_pid_file cluster1:0 2020-04-10T07:11:14.308 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:11:14.308 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:11:14.308 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:11:14.309 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:11:14.309 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:11:14.309 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:11:14.309 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:11:14.309 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:11:14.309 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:11:14.309 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:11:14.310 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:11:14.310 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:11:14.310 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:11:14.310 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 pid file 2020-04-10T07:11:14.321 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.pid 2020-04-10T07:11:14.321 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cat /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.pid 2020-04-10T07:11:14.322 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ pid=15642 2020-04-10T07:11:14.322 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n 15642 ] 2020-04-10T07:11:14.322 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ kill 15642 2020-04-10T07:11:14.322 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:11:15.323 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ps auxww 2020-04-10T07:11:15.323 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ awk -v pid=15642 $2 == pid {print; exit 1} 2020-04-10T07:11:15.339 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ break 2020-04-10T07:11:15.340 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ps auxww 2020-04-10T07:11:15.340 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ awk -v pid=15642 $2 == pid {print; exit 1} 2020-04-10T07:11:15.357 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster1 2020-04-10T07:11:15.357 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:11:15.357 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:11:15.358 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:11:15.358 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:11:15.358 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:11:15.358 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:11:15.358 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:11:15.358 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:11:15.358 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:11:15.359 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:11:15.359 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:11:15.359 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:11:15.359 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:11:15.359 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:11:15.359 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:11:15.378 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:11:15.378 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rm -f /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:11:15.380 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:0 cluster2 2020-04-10T07:11:15.380 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:0 2020-04-10T07:11:15.381 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:11:15.381 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:11:15.381 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2020-04-10T07:11:15.381 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:11:15.381 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:11:15.381 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:11:15.381 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:11:15.382 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:11:15.382 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:11:15.383 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:11:15.384 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:11:15.384 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:11:15.384 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:11:15.384 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2020-04-10T07:11:15.402 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:11:15.402 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rm -f /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2020-04-10T07:11:15.402 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_pid_file cluster1:0 2020-04-10T07:11:15.402 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:11:15.402 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:11:15.403 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:11:15.403 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:11:15.403 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:11:15.403 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:11:15.403 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:11:15.403 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:11:15.403 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:11:15.404 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:11:15.404 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:11:15.404 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:11:15.404 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:11:15.404 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 pid file 2020-04-10T07:11:15.420 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.pid 2020-04-10T07:11:15.420 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rm -f /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.pid 2020-04-10T07:11:15.421 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ stop_mirror cluster1:1 2020-04-10T07:11:15.421 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:1 2020-04-10T07:11:15.422 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local sig= 2020-04-10T07:11:15.422 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 2020-04-10T07:11:15.422 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pid 2020-04-10T07:11:15.423 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_pid_file cluster1:1 2020-04-10T07:11:15.424 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:1 2020-04-10T07:11:15.424 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:11:15.424 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 cluster instance 2020-04-10T07:11:15.424 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:11:15.424 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:11:15.424 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:11:15.425 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:11:15.425 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:11:15.425 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:11:15.425 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:11:15.425 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:11:15.425 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:11:15.425 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:11:15.426 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 pid file 2020-04-10T07:11:15.441 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.pid 2020-04-10T07:11:15.442 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cat /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.pid 2020-04-10T07:11:15.442 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ pid=15673 2020-04-10T07:11:15.442 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n 15673 ] 2020-04-10T07:11:15.443 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ kill 15673 2020-04-10T07:11:15.443 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:11:16.444 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ps auxww 2020-04-10T07:11:16.444 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ awk -v pid=15673 $2 == pid {print; exit 1} 2020-04-10T07:11:16.463 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ break 2020-04-10T07:11:16.463 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ps auxww 2020-04-10T07:11:16.464 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ awk -v pid=15673 $2 == pid {print; exit 1} 2020-04-10T07:11:16.479 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:1 cluster1 2020-04-10T07:11:16.479 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:1 2020-04-10T07:11:16.479 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:11:16.479 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:11:16.479 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2020-04-10T07:11:16.479 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:11:16.480 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:11:16.480 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:11:16.480 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:11:16.480 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:11:16.480 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:11:16.480 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:11:16.481 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:11:16.481 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:11:16.481 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:11:16.481 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2020-04-10T07:11:16.495 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:11:16.495 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rm -f /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:11:16.497 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:1 cluster2 2020-04-10T07:11:16.497 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:1 2020-04-10T07:11:16.497 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:11:16.498 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:11:16.498 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2020-04-10T07:11:16.498 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:11:16.498 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:11:16.498 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:11:16.498 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:11:16.498 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:11:16.499 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:11:16.499 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:11:16.499 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:11:16.499 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:11:16.499 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:11:16.500 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2020-04-10T07:11:16.517 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:11:16.518 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rm -f /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2020-04-10T07:11:16.519 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_pid_file cluster1:1 2020-04-10T07:11:16.520 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:1 2020-04-10T07:11:16.520 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:11:16.520 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 cluster instance 2020-04-10T07:11:16.520 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:11:16.520 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:11:16.520 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:11:16.520 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:11:16.521 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:11:16.521 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:11:16.521 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:11:16.521 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:11:16.521 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:11:16.521 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:11:16.521 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 pid file 2020-04-10T07:11:16.540 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.pid 2020-04-10T07:11:16.541 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rm -f /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.pid 2020-04-10T07:11:16.542 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ stop_mirror cluster1:2 2020-04-10T07:11:16.542 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:2 2020-04-10T07:11:16.542 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local sig= 2020-04-10T07:11:16.542 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 2020-04-10T07:11:16.542 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pid 2020-04-10T07:11:16.543 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_pid_file cluster1:2 2020-04-10T07:11:16.543 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:2 2020-04-10T07:11:16.544 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:11:16.544 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 cluster instance 2020-04-10T07:11:16.544 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:11:16.544 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:11:16.544 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:11:16.544 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:11:16.545 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:11:16.545 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:11:16.545 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:11:16.545 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:11:16.545 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:11:16.545 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:11:16.545 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 pid file 2020-04-10T07:11:16.568 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.pid 2020-04-10T07:11:16.568 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cat /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.pid 2020-04-10T07:11:16.568 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ pid=15704 2020-04-10T07:11:16.569 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n 15704 ] 2020-04-10T07:11:16.569 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ kill 15704 2020-04-10T07:11:16.569 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:11:17.570 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ps auxww 2020-04-10T07:11:17.571 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ awk -v pid=15704 $2 == pid {print; exit 1} 2020-04-10T07:11:17.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ break 2020-04-10T07:11:17.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ps auxww 2020-04-10T07:11:17.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ awk -v pid=15704 $2 == pid {print; exit 1} 2020-04-10T07:11:17.600 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:2 cluster1 2020-04-10T07:11:17.600 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:2 2020-04-10T07:11:17.600 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:11:17.601 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:11:17.601 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2020-04-10T07:11:17.601 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:11:17.601 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:11:17.601 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:11:17.601 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:11:17.601 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:11:17.602 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:11:17.602 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:11:17.602 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:11:17.602 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:11:17.602 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:11:17.602 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2020-04-10T07:11:17.619 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:11:17.619 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rm -f /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:11:17.621 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:2 cluster2 2020-04-10T07:11:17.622 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:2 2020-04-10T07:11:17.622 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:11:17.622 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:11:17.622 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2020-04-10T07:11:17.622 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:11:17.622 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:11:17.623 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:11:17.623 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:11:17.623 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:11:17.623 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:11:17.623 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:11:17.623 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:11:17.623 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:11:17.623 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:11:17.624 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2020-04-10T07:11:17.638 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:11:17.638 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rm -f /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2020-04-10T07:11:17.641 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_pid_file cluster1:2 2020-04-10T07:11:17.641 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:2 2020-04-10T07:11:17.642 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:11:17.642 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 cluster instance 2020-04-10T07:11:17.642 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:11:17.642 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:11:17.642 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:11:17.642 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:11:17.642 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:11:17.643 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:11:17.643 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:11:17.643 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:11:17.643 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:11:17.643 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:11:17.643 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 pid file 2020-04-10T07:11:17.655 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.pid 2020-04-10T07:11:17.655 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rm -f /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.pid 2020-04-10T07:11:17.658 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ stop_mirror cluster1:3 2020-04-10T07:11:17.658 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:3 2020-04-10T07:11:17.659 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local sig= 2020-04-10T07:11:17.659 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 2020-04-10T07:11:17.659 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pid 2020-04-10T07:11:17.659 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_pid_file cluster1:3 2020-04-10T07:11:17.659 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:3 2020-04-10T07:11:17.659 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:11:17.659 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:3 cluster instance 2020-04-10T07:11:17.660 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:3 2020-04-10T07:11:17.660 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:11:17.660 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:11:17.660 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:11:17.660 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:11:17.660 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 3 = cluster1:3 ] 2020-04-10T07:11:17.660 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:11:17.660 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:11:17.661 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=3 2020-04-10T07:11:17.661 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:11:17.661 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.3 pid file 2020-04-10T07:11:17.680 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.pid 2020-04-10T07:11:17.680 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cat /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.pid 2020-04-10T07:11:17.680 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ pid=15803 2020-04-10T07:11:17.681 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n 15803 ] 2020-04-10T07:11:17.681 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ kill 15803 2020-04-10T07:11:17.681 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:11:18.685 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ps auxww 2020-04-10T07:11:18.686 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ awk -v pid=15803 $2 == pid {print; exit 1} 2020-04-10T07:11:18.705 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ break 2020-04-10T07:11:18.705 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ps auxww 2020-04-10T07:11:18.705 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ awk -v pid=15803 $2 == pid {print; exit 1} 2020-04-10T07:11:18.713 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:3 cluster1 2020-04-10T07:11:18.713 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:3 2020-04-10T07:11:18.714 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:11:18.714 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:11:18.714 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:3 local_cluster instance 2020-04-10T07:11:18.714 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:3 2020-04-10T07:11:18.714 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:11:18.714 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:11:18.714 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:11:18.715 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:11:18.715 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 3 = cluster1:3 ] 2020-04-10T07:11:18.715 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:11:18.715 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:11:18.715 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=3 2020-04-10T07:11:18.715 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:11:18.715 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.3 admin socket 2020-04-10T07:11:18.735 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:11:18.735 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rm -f /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:11:18.736 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster1:3 cluster2 2020-04-10T07:11:18.736 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster1:3 2020-04-10T07:11:18.736 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:11:18.737 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:11:18.737 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:3 local_cluster instance 2020-04-10T07:11:18.737 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:3 2020-04-10T07:11:18.737 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:11:18.737 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:11:18.737 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:11:18.737 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:11:18.737 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 3 = cluster1:3 ] 2020-04-10T07:11:18.738 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster1 2020-04-10T07:11:18.738 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster1 2020-04-10T07:11:18.738 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=3 2020-04-10T07:11:18.739 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:11:18.739 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.3 admin socket 2020-04-10T07:11:18.757 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:11:18.757 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rm -f /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2020-04-10T07:11:18.759 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_pid_file cluster1:3 2020-04-10T07:11:18.760 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:3 2020-04-10T07:11:18.760 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:11:18.760 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:3 cluster instance 2020-04-10T07:11:18.760 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:3 2020-04-10T07:11:18.760 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:11:18.760 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:11:18.761 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:11:18.761 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:11:18.761 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 3 = cluster1:3 ] 2020-04-10T07:11:18.761 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:11:18.761 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:11:18.761 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=3 2020-04-10T07:11:18.761 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:11:18.761 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.3 pid file 2020-04-10T07:11:18.778 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.pid 2020-04-10T07:11:18.778 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rm -f /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.pid 2020-04-10T07:11:18.780 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ stop_mirrors cluster2 2020-04-10T07:11:18.780 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:11:18.780 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local sig= 2020-04-10T07:11:18.780 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:11:18.782 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ stop_mirror cluster2:0 2020-04-10T07:11:18.782 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2:0 2020-04-10T07:11:18.782 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local sig= 2020-04-10T07:11:18.782 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 2020-04-10T07:11:18.782 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pid 2020-04-10T07:11:18.782 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_pid_file cluster2:0 2020-04-10T07:11:18.783 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2:0 2020-04-10T07:11:18.783 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:11:18.783 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:0 cluster instance 2020-04-10T07:11:18.783 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:0 2020-04-10T07:11:18.783 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:11:18.783 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:11:18.783 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:11:18.783 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:11:18.784 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster2:0 ] 2020-04-10T07:11:18.784 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster2 2020-04-10T07:11:18.784 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:11:18.784 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:11:18.784 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:11:18.784 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster2 --name client.mirror.0 pid file 2020-04-10T07:11:18.802 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster2-client.mirror.0.pid 2020-04-10T07:11:18.802 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cat /var/run/ceph/rbd-mirror.cluster2-client.mirror.0.pid 2020-04-10T07:11:18.802 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ pid=19135 2020-04-10T07:11:18.803 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n 19135 ] 2020-04-10T07:11:18.803 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ kill 19135 2020-04-10T07:11:18.803 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:11:19.804 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ps auxww 2020-04-10T07:11:19.804 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ awk -v pid=19135 $2 == pid {print; exit 1} 2020-04-10T07:11:19.821 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ break 2020-04-10T07:11:19.822 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ps auxww 2020-04-10T07:11:19.822 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ awk -v pid=19135 $2 == pid {print; exit 1} 2020-04-10T07:11:19.841 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster2:0 cluster1 2020-04-10T07:11:19.841 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster2:0 2020-04-10T07:11:19.841 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:11:19.841 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:11:19.841 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:0 local_cluster instance 2020-04-10T07:11:19.842 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:0 2020-04-10T07:11:19.842 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:11:19.842 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:11:19.842 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:11:19.842 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:11:19.842 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster2:0 ] 2020-04-10T07:11:19.842 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster2 2020-04-10T07:11:19.843 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster2 2020-04-10T07:11:19.843 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:11:19.843 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:11:19.843 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster2 --name client.mirror.0 admin socket 2020-04-10T07:11:19.861 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster2-client.mirror.0.asok 2020-04-10T07:11:19.862 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rm -f /var/run/ceph/rbd-mirror.cluster2-client.mirror.0.asok 2020-04-10T07:11:19.864 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster2:0 cluster2 2020-04-10T07:11:19.864 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster2:0 2020-04-10T07:11:19.864 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:11:19.865 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:11:19.865 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:0 local_cluster instance 2020-04-10T07:11:19.865 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:0 2020-04-10T07:11:19.865 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:11:19.865 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:11:19.865 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:11:19.865 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:11:19.865 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster2:0 ] 2020-04-10T07:11:19.866 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster2 2020-04-10T07:11:19.866 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster2 2020-04-10T07:11:19.866 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:11:19.866 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:11:19.866 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster2 --name client.mirror.0 admin socket 2020-04-10T07:11:19.889 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster2-client.mirror.0.asok 2020-04-10T07:11:19.889 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rm -f /var/run/ceph/rbd-mirror.cluster2-client.mirror.0.asok 2020-04-10T07:11:19.891 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_pid_file cluster2:0 2020-04-10T07:11:19.891 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2:0 2020-04-10T07:11:19.891 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:11:19.891 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:0 cluster instance 2020-04-10T07:11:19.891 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:0 2020-04-10T07:11:19.891 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:11:19.891 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:11:19.892 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:11:19.892 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:11:19.892 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster2:0 ] 2020-04-10T07:11:19.892 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster2 2020-04-10T07:11:19.892 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:11:19.892 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:11:19.893 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:11:19.893 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster2 --name client.mirror.0 pid file 2020-04-10T07:11:19.906 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster2-client.mirror.0.pid 2020-04-10T07:11:19.906 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rm -f /var/run/ceph/rbd-mirror.cluster2-client.mirror.0.pid 2020-04-10T07:11:19.909 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ stop_mirror cluster2:1 2020-04-10T07:11:19.909 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2:1 2020-04-10T07:11:19.909 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local sig= 2020-04-10T07:11:19.909 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 2020-04-10T07:11:19.909 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pid 2020-04-10T07:11:19.910 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_pid_file cluster2:1 2020-04-10T07:11:19.910 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2:1 2020-04-10T07:11:19.910 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:11:19.910 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:1 cluster instance 2020-04-10T07:11:19.910 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:1 2020-04-10T07:11:19.910 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:11:19.910 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:11:19.911 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:11:19.911 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:11:19.911 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster2:1 ] 2020-04-10T07:11:19.911 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster2 2020-04-10T07:11:19.911 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:11:19.911 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:11:19.911 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:11:19.912 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster2 --name client.mirror.1 pid file 2020-04-10T07:11:19.922 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster2-client.mirror.1.pid 2020-04-10T07:11:19.922 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cat /var/run/ceph/rbd-mirror.cluster2-client.mirror.1.pid 2020-04-10T07:11:19.923 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ pid=19166 2020-04-10T07:11:19.923 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n 19166 ] 2020-04-10T07:11:19.923 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ kill 19166 2020-04-10T07:11:19.923 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:11:20.924 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ps auxww 2020-04-10T07:11:20.924 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ awk -v pid=19166 $2 == pid {print; exit 1} 2020-04-10T07:11:20.944 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ break 2020-04-10T07:11:20.944 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ps auxww 2020-04-10T07:11:20.944 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ awk -v pid=19166 $2 == pid {print; exit 1} 2020-04-10T07:11:20.964 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster2:1 cluster1 2020-04-10T07:11:20.964 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster2:1 2020-04-10T07:11:20.964 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1 2020-04-10T07:11:20.964 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:11:20.965 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:1 local_cluster instance 2020-04-10T07:11:20.965 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:1 2020-04-10T07:11:20.965 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:11:20.965 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:11:20.965 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:11:20.966 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:11:20.966 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster2:1 ] 2020-04-10T07:11:20.966 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster2 2020-04-10T07:11:20.966 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster2 2020-04-10T07:11:20.967 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:11:20.967 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:11:20.967 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster2 --name client.mirror.1 admin socket 2020-04-10T07:11:20.987 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster2-client.mirror.1.asok 2020-04-10T07:11:20.987 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rm -f /var/run/ceph/rbd-mirror.cluster2-client.mirror.1.asok 2020-04-10T07:11:20.989 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster2:1 cluster2 2020-04-10T07:11:20.989 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster2:1 2020-04-10T07:11:20.989 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:11:20.989 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:11:20.989 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:1 local_cluster instance 2020-04-10T07:11:20.990 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:1 2020-04-10T07:11:20.990 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:11:20.990 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:11:20.990 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:11:20.990 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:11:20.990 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster2:1 ] 2020-04-10T07:11:20.990 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster2 2020-04-10T07:11:20.991 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster2 2020-04-10T07:11:20.991 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:11:20.991 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:11:20.991 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster2 --name client.mirror.1 admin socket 2020-04-10T07:11:21.003 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster2-client.mirror.1.asok 2020-04-10T07:11:21.003 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rm -f /var/run/ceph/rbd-mirror.cluster2-client.mirror.1.asok 2020-04-10T07:11:21.005 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_pid_file cluster2:1 2020-04-10T07:11:21.005 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2:1 2020-04-10T07:11:21.005 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:11:21.005 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:1 cluster instance 2020-04-10T07:11:21.005 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:1 2020-04-10T07:11:21.006 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:11:21.006 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:11:21.006 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:11:21.006 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:11:21.006 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster2:1 ] 2020-04-10T07:11:21.006 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster2 2020-04-10T07:11:21.006 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:11:21.007 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:11:21.007 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:11:21.007 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster2 --name client.mirror.1 pid file 2020-04-10T07:11:21.023 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster2-client.mirror.1.pid 2020-04-10T07:11:21.024 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rm -f /var/run/ceph/rbd-mirror.cluster2-client.mirror.1.pid 2020-04-10T07:11:21.026 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ stop_mirror cluster2:2 2020-04-10T07:11:21.026 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2:2 2020-04-10T07:11:21.026 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local sig= 2020-04-10T07:11:21.026 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ test -n 2020-04-10T07:11:21.026 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pid 2020-04-10T07:11:21.027 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_pid_file cluster2:2 2020-04-10T07:11:21.027 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2:2 2020-04-10T07:11:21.027 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:11:21.027 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:2 cluster instance 2020-04-10T07:11:21.027 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:2 2020-04-10T07:11:21.027 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:11:21.027 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:11:21.028 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:11:21.028 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:11:21.028 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster2:2 ] 2020-04-10T07:11:21.028 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster2 2020-04-10T07:11:21.029 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:11:21.029 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:11:21.029 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:11:21.029 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster2 --name client.mirror.2 pid file 2020-04-10T07:11:21.044 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster2-client.mirror.2.pid 2020-04-10T07:11:21.044 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cat /var/run/ceph/rbd-mirror.cluster2-client.mirror.2.pid 2020-04-10T07:11:21.045 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ pid=19197 2020-04-10T07:11:21.045 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -n 19197 ] 2020-04-10T07:11:21.045 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ kill 19197 2020-04-10T07:11:21.046 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 1 2020-04-10T07:11:22.046 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ps auxww 2020-04-10T07:11:22.047 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ awk -v pid=19197 $2 == pid {print; exit 1} 2020-04-10T07:11:22.064 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:ubuntu 19197 0.2 0.1 711736 34820 ? Ssl 07:08 0:00 rbd-mirror --cluster cluster2 --id mirror.2 --rbd-mirror-delete-retry-interval=5 --rbd-mirror-image-state-check-interval=5 --rbd-mirror-journal-poll-age=1 --rbd-mirror-pool-replayers-refresh-interval=5 --debug-rbd=30 --debug-journaler=30 --debug-rbd_mirror=30 --daemonize=true 2020-04-10T07:11:22.064 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 2 2020-04-10T07:11:24.065 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ + awkps -v pid=19197 auxww $2 == pid {print; exit 1} 2020-04-10T07:11:24.066 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr: 2020-04-10T07:11:24.078 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:ubuntu 19197 0.2 0.1 803940 38560 ? Ssl 07:08 0:00 rbd-mirror --cluster cluster2 --id mirror.2 --rbd-mirror-delete-retry-interval=5 --rbd-mirror-image-state-check-interval=5 --rbd-mirror-journal-poll-age=1 --rbd-mirror-pool-replayers-refresh-interval=5 --debug-rbd=30 --debug-journaler=30 --debug-rbd_mirror=30 --daemonize=true 2020-04-10T07:11:24.079 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 4 2020-04-10T07:11:28.078 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ps auxww 2020-04-10T07:11:28.078 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ awk -v pid=19197 $2 == pid {print; exit 1} 2020-04-10T07:11:28.095 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:ubuntu 19197 0.2 0.1 803940 38824 ? Ssl 07:08 0:00 rbd-mirror --cluster cluster2 --id mirror.2 --rbd-mirror-delete-retry-interval=5 --rbd-mirror-image-state-check-interval=5 --rbd-mirror-journal-poll-age=1 --rbd-mirror-pool-replayers-refresh-interval=5 --debug-rbd=30 --debug-journaler=30 --debug-rbd_mirror=30 --daemonize=true 2020-04-10T07:11:28.096 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 8 2020-04-10T07:11:36.098 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ps auxww 2020-04-10T07:11:36.098 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ awk -v pid=19197 $2 == pid {print; exit 1} 2020-04-10T07:11:36.115 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:ubuntu 19197 0.2 0.1 803940 39352 ? Ssl 07:08 0:00 rbd-mirror --cluster cluster2 --id mirror.2 --rbd-mirror-delete-retry-interval=5 --rbd-mirror-image-state-check-interval=5 --rbd-mirror-journal-poll-age=1 --rbd-mirror-pool-replayers-refresh-interval=5 --debug-rbd=30 --debug-journaler=30 --debug-rbd_mirror=30 --daemonize=true 2020-04-10T07:11:36.116 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 16 2020-04-10T07:11:52.118 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ps auxww 2020-04-10T07:11:52.119 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ awk -v pid=19197 $2 == pid {print; exit 1} 2020-04-10T07:11:52.136 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:ubuntu 19197 0.2 0.1 803940 40672 ? Ssl 07:08 0:00 rbd-mirror --cluster cluster2 --id mirror.2 --rbd-mirror-delete-retry-interval=5 --rbd-mirror-image-state-check-interval=5 --rbd-mirror-journal-poll-age=1 --rbd-mirror-pool-replayers-refresh-interval=5 --debug-rbd=30 --debug-journaler=30 --debug-rbd_mirror=30 --daemonize=true 2020-04-10T07:11:52.136 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ sleep 32 2020-04-10T07:12:24.138 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ps auxww 2020-04-10T07:12:24.139 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ awk -v pid=19197 $2 == pid {print; exit 1} 2020-04-10T07:12:24.154 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:ubuntu 19197 0.2 0.1 803940 42784 ? Ssl 07:08 0:00 rbd-mirror --cluster cluster2 --id mirror.2 --rbd-mirror-delete-retry-interval=5 --rbd-mirror-image-state-check-interval=5 --rbd-mirror-journal-poll-age=1 --rbd-mirror-pool-replayers-refresh-interval=5 --debug-rbd=30 --debug-journaler=30 --debug-rbd_mirror=30 --daemonize=true 2020-04-10T07:12:24.154 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ps auxww 2020-04-10T07:12:24.155 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ awk -v pid=19197 $2 == pid {print; exit 1} 2020-04-10T07:12:24.164 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:ubuntu 19197 0.2 0.1 803940 42784 ? Ssl 07:08 0:00 rbd-mirror --cluster cluster2 --id mirror.2 --rbd-mirror-delete-retry-interval=5 --rbd-mirror-image-state-check-interval=5 --rbd-mirror-journal-poll-age=1 --rbd-mirror-pool-replayers-refresh-interval=5 --debug-rbd=30 --debug-journaler=30 --debug-rbd_mirror=30 --daemonize=true 2020-04-10T07:12:24.165 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cleanup 1 2020-04-10T07:12:24.165 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local error_code=1 2020-04-10T07:12:24.165 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set +e 2020-04-10T07:12:24.166 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 -ne 0 ] 2020-04-10T07:12:24.166 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ status 2020-04-10T07:12:24.166 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster daemon image_pool image_ns image 2020-04-10T07:12:24.166 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1 status 2020-04-10T07:12:24.166 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= ceph --cluster cluster1 -s 2020-04-10T07:12:24.166 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster1 status 2020-04-10T07:12:24.581 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: cluster: 2020-04-10T07:12:24.581 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: id: 5d7eff56-55fb-4da2-8b7d-3ec54a26cfc2 2020-04-10T07:12:24.581 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: health: HEALTH_OK 2020-04-10T07:12:24.581 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:24.581 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: services: 2020-04-10T07:12:24.582 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mon: 1 daemons, quorum a (age 8m) 2020-04-10T07:12:24.582 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mgr: x(active, since 8m) 2020-04-10T07:12:24.582 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: osd: 3 osds: 3 up (since 8m), 3 in (since 8m) 2020-04-10T07:12:24.582 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:24.582 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: data: 2020-04-10T07:12:24.582 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: pools: 4 pools, 137 pgs 2020-04-10T07:12:24.582 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: objects: 524 objects, 720 MiB 2020-04-10T07:12:24.582 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: usage: 3.0 GiB used, 267 GiB / 270 GiB avail 2020-04-10T07:12:24.583 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: pgs: 137 active+clean 2020-04-10T07:12:24.583 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:24.583 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: io: 2020-04-10T07:12:24.583 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: client: 18 KiB/s rd, 2.2 KiB/s wr, 17 op/s rd, 2 op/s wr 2020-04-10T07:12:24.583 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:24.591 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= ceph --cluster cluster1 service dump 2020-04-10T07:12:24.916 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:{ 2020-04-10T07:12:24.916 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "epoch": 8, 2020-04-10T07:12:24.916 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "modified": "2020-04-10T07:11:18.743912+0000", 2020-04-10T07:12:24.916 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "services": {} 2020-04-10T07:12:24.917 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:} 2020-04-10T07:12:24.928 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= ceph --cluster cluster1 service status 2020-04-10T07:12:25.236 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:{} 2020-04-10T07:12:25.247 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:25.248 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster1 mirror images 2020-04-10T07:12:25.248 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:25.248 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1 mirror images 2020-04-10T07:12:25.248 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror --namespace ls -l 2020-04-10T07:12:25.354 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:NAME SIZE PARENT FMT PROT LOCK 2020-04-10T07:12:25.354 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test 128 MiB 2 2020-04-10T07:12:25.354 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test1 128 MiB 2 2020-04-10T07:12:25.355 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test2 128 MiB 2 2020-04-10T07:12:25.355 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test2@snap1 128 MiB 2 2020-04-10T07:12:25.355 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test2@snap2 128 MiB 2 2020-04-10T07:12:25.355 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test2@snap_rename_20 128 MiB 2 2020-04-10T07:12:25.355 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test_clone 128 MiB mirror_parent/test_parent@snap 2 2020-04-10T07:12:25.355 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test_clone1 128 MiB mirror_parent/test_parent@snap 2 2020-04-10T07:12:25.355 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test_clone_v1 128 MiB mirror_parent/test_parent@snap 2 2020-04-10T07:12:25.356 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test_data_pool 128 MiB 2 2020-04-10T07:12:25.356 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test_data_pool@snap1 128 MiB 2 2020-04-10T07:12:25.356 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test_data_pool@snap2 128 MiB 2 2020-04-10T07:12:25.356 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test_force_promote 128 MiB 2 2020-04-10T07:12:25.360 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:25.360 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1 mirror mirror pool info 2020-04-10T07:12:25.360 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror --namespace mirror pool info 2020-04-10T07:12:25.360 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:25.360 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster1 mirror mirror pool info 2020-04-10T07:12:25.395 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Mode: image 2020-04-10T07:12:25.395 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Site Name: cluster1-DNE 2020-04-10T07:12:25.396 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:25.396 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Peer Sites: 2020-04-10T07:12:25.396 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:25.396 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:UUID: 2571f49c-d767-4112-ae12-d8b71263d58b 2020-04-10T07:12:25.396 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Name: cluster2-DNE 2020-04-10T07:12:25.396 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Mirror UUID: 8253fde2-86f8-443a-b4bf-ec7dae2fafa3 2020-04-10T07:12:25.397 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Direction: rx-tx 2020-04-10T07:12:25.397 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Client: client.mirror 2020-04-10T07:12:25.399 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:25.399 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster1 mirror mirror pool status 2020-04-10T07:12:25.400 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:25.400 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1 mirror mirror pool status 2020-04-10T07:12:25.400 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster1 -p mirror --namespace mirror pool status --verbose 2020-04-10T07:12:25.440 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:health: UNKNOWN 2020-04-10T07:12:25.440 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:daemon health: UNKNOWN 2020-04-10T07:12:25.440 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image health: OK 2020-04-10T07:12:25.440 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:images: 8 total 2020-04-10T07:12:25.441 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 1 replaying 2020-04-10T07:12:25.441 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 7 stopped 2020-04-10T07:12:25.441 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:25.441 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:DAEMONS 2020-04-10T07:12:25.442 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: none 2020-04-10T07:12:25.442 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:25.497 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:IMAGES 2020-04-10T07:12:25.498 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test: 2020-04-10T07:12:25.498 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: global_id: a9af7d9e-1404-4730-9e18-c0418a915346 2020-04-10T07:12:25.498 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: down+stopped 2020-04-10T07:12:25.498 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: stopped 2020-04-10T07:12:25.498 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 07:11:16 2020-04-10T07:12:25.498 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: peer_sites: 2020-04-10T07:12:25.499 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: name: cluster2-DNE 2020-04-10T07:12:25.499 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+stopped 2020-04-10T07:12:25.499 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: local image is primary 2020-04-10T07:12:25.499 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 07:12:21 2020-04-10T07:12:25.499 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:25.499 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test1: 2020-04-10T07:12:25.499 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: global_id: 56419286-13e9-49f6-a20d-2abe6ede083e 2020-04-10T07:12:25.500 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: down+stopped 2020-04-10T07:12:25.500 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: stopped 2020-04-10T07:12:25.500 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 07:11:14 2020-04-10T07:12:25.500 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: peer_sites: 2020-04-10T07:12:25.500 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: name: cluster2-DNE 2020-04-10T07:12:25.500 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+stopped 2020-04-10T07:12:25.500 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: local image is primary 2020-04-10T07:12:25.500 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 07:12:21 2020-04-10T07:12:25.501 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:25.501 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test2: 2020-04-10T07:12:25.501 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: global_id: b583e294-8b69-4e57-9966-783595c454d6 2020-04-10T07:12:25.502 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: down+stopped 2020-04-10T07:12:25.502 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: stopped 2020-04-10T07:12:25.502 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 07:11:15 2020-04-10T07:12:25.502 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: peer_sites: 2020-04-10T07:12:25.502 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: name: cluster2-DNE 2020-04-10T07:12:25.502 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+stopped 2020-04-10T07:12:25.502 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: local image is primary 2020-04-10T07:12:25.503 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 07:12:21 2020-04-10T07:12:25.511 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:25.512 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test_clone: 2020-04-10T07:12:25.512 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: global_id: 003078e2-ed6c-4f36-8bc5-ce0c44ef25d5 2020-04-10T07:12:25.512 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: down+stopped 2020-04-10T07:12:25.512 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: stopped 2020-04-10T07:12:25.512 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 07:11:14 2020-04-10T07:12:25.512 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: peer_sites: 2020-04-10T07:12:25.517 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: name: cluster2-DNE 2020-04-10T07:12:25.517 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+stopped 2020-04-10T07:12:25.517 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: local image is primary 2020-04-10T07:12:25.517 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 07:12:21 2020-04-10T07:12:25.518 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:25.518 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test_clone1: 2020-04-10T07:12:25.518 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: global_id: 4b1961cb-5406-443f-90f1-3627ac4ffffd 2020-04-10T07:12:25.519 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: down+stopped 2020-04-10T07:12:25.519 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: local image is primary 2020-04-10T07:12:25.519 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 07:11:15 2020-04-10T07:12:25.519 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: peer_sites: 2020-04-10T07:12:25.519 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: name: cluster2-DNE 2020-04-10T07:12:25.519 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+replaying 2020-04-10T07:12:25.519 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: replaying, {"replay_state": "idle", "remote_snapshot_timestamp": 1586502598, "local_snapshot_timestamp": 1586502598} 2020-04-10T07:12:25.520 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 07:12:06 2020-04-10T07:12:25.520 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:25.520 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test_clone_v1: 2020-04-10T07:12:25.520 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: global_id: 70d86755-f73a-4bbe-9493-1b79a191cc0c 2020-04-10T07:12:25.521 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: down+stopped 2020-04-10T07:12:25.521 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: stopped 2020-04-10T07:12:25.521 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 07:11:16 2020-04-10T07:12:25.521 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: peer_sites: 2020-04-10T07:12:25.521 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: name: cluster2-DNE 2020-04-10T07:12:25.521 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+stopped 2020-04-10T07:12:25.522 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: local image is primary 2020-04-10T07:12:25.522 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 07:12:21 2020-04-10T07:12:25.522 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:25.522 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test_data_pool: 2020-04-10T07:12:25.522 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: global_id: a8183526-4331-4f0d-af4a-7c93c501e927 2020-04-10T07:12:25.522 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: down+stopped 2020-04-10T07:12:25.522 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: stopped 2020-04-10T07:12:25.523 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 07:11:14 2020-04-10T07:12:25.523 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: peer_sites: 2020-04-10T07:12:25.523 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: name: cluster2-DNE 2020-04-10T07:12:25.523 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+stopped 2020-04-10T07:12:25.523 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: local image is primary 2020-04-10T07:12:25.523 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 07:12:21 2020-04-10T07:12:25.523 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:25.523 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test_force_promote: 2020-04-10T07:12:25.524 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: global_id: b192e243-6abc-405a-b798-b60b9d74691e 2020-04-10T07:12:25.524 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: down+stopped 2020-04-10T07:12:25.524 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: local image is primary 2020-04-10T07:12:25.524 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 07:11:16 2020-04-10T07:12:25.524 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: peer_sites: 2020-04-10T07:12:25.524 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: name: cluster2-DNE 2020-04-10T07:12:25.524 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+stopped 2020-04-10T07:12:25.526 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: local image is primary 2020-04-10T07:12:25.526 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 07:12:21 2020-04-10T07:12:25.527 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:25.527 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:25.527 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror --namespace ls 2020-04-10T07:12:25.567 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test info 2020-04-10T07:12:25.567 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test info 2020-04-10T07:12:25.567 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror --namespace info test 2020-04-10T07:12:25.639 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:rbd image 'test': 2020-04-10T07:12:25.639 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: size 128 MiB in 32 objects 2020-04-10T07:12:25.640 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: order 22 (4 MiB objects) 2020-04-10T07:12:25.640 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: snapshot_count: 14 2020-04-10T07:12:25.640 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: id: 114bcee179da 2020-04-10T07:12:25.640 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: block_name_prefix: rbd_data.114bcee179da 2020-04-10T07:12:25.641 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: format: 2 2020-04-10T07:12:25.641 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: features: layering, exclusive-lock, journaling, non-primary 2020-04-10T07:12:25.641 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: op_features: 2020-04-10T07:12:25.641 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: flags: 2020-04-10T07:12:25.641 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: create_timestamp: Fri Apr 10 07:08:05 2020 2020-04-10T07:12:25.641 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: access_timestamp: Fri Apr 10 07:09:20 2020 2020-04-10T07:12:25.641 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: modify_timestamp: Fri Apr 10 07:08:05 2020 2020-04-10T07:12:25.642 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: journal: 114bcee179da 2020-04-10T07:12:25.642 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring state: enabled 2020-04-10T07:12:25.642 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring mode: snapshot 2020-04-10T07:12:25.642 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring global id: a9af7d9e-1404-4730-9e18-c0418a915346 2020-04-10T07:12:25.642 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring primary: false 2020-04-10T07:12:25.643 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:25.643 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test journal status 2020-04-10T07:12:25.644 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:25.644 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test journal status 2020-04-10T07:12:25.644 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror --namespace journal status --image test 2020-04-10T07:12:25.727 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:minimum_set: 0 2020-04-10T07:12:25.727 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:active_set: 0 2020-04-10T07:12:25.727 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:registered clients: 2020-04-10T07:12:25.727 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: [id=, commit_position=[positions=[[object_number=1, tag_tid=7, entry_tid=1], [object_number=0, tag_tid=7, entry_tid=0], [object_number=3, tag_tid=4, entry_tid=99], [object_number=2, tag_tid=4, entry_tid=98]]], state=connected] 2020-04-10T07:12:25.731 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:25.731 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test1 info 2020-04-10T07:12:25.731 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:25.732 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test1 info 2020-04-10T07:12:25.732 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror --namespace info test1 2020-04-10T07:12:25.795 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:rbd image 'test1': 2020-04-10T07:12:25.796 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: size 128 MiB in 32 objects 2020-04-10T07:12:25.796 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: order 22 (4 MiB objects) 2020-04-10T07:12:25.796 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: snapshot_count: 2 2020-04-10T07:12:25.796 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: id: 113f5aed03a0 2020-04-10T07:12:25.797 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: block_name_prefix: rbd_data.113f5aed03a0 2020-04-10T07:12:25.797 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: format: 2 2020-04-10T07:12:25.797 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: features: layering, exclusive-lock, journaling, non-primary 2020-04-10T07:12:25.797 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: op_features: 2020-04-10T07:12:25.797 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: flags: 2020-04-10T07:12:25.798 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: create_timestamp: Fri Apr 10 07:06:41 2020 2020-04-10T07:12:25.798 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: access_timestamp: Fri Apr 10 07:06:41 2020 2020-04-10T07:12:25.798 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: modify_timestamp: Fri Apr 10 07:06:41 2020 2020-04-10T07:12:25.798 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: journal: 113f5aed03a0 2020-04-10T07:12:25.799 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring state: enabled 2020-04-10T07:12:25.799 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring mode: snapshot 2020-04-10T07:12:25.799 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring global id: 56419286-13e9-49f6-a20d-2abe6ede083e 2020-04-10T07:12:25.799 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring primary: false 2020-04-10T07:12:25.800 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:25.800 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test1 journal status 2020-04-10T07:12:25.801 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:25.801 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test1 journal status 2020-04-10T07:12:25.801 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror --namespace journal status --image test1 2020-04-10T07:12:25.873 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:minimum_set: 0 2020-04-10T07:12:25.874 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:active_set: 0 2020-04-10T07:12:25.874 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:registered clients: 2020-04-10T07:12:25.874 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: [id=, commit_position=[positions=[]], state=connected] 2020-04-10T07:12:25.877 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:25.877 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test2 info 2020-04-10T07:12:25.877 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:25.878 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test2 info 2020-04-10T07:12:25.878 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror --namespace info test2 2020-04-10T07:12:25.940 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:rbd image 'test2': 2020-04-10T07:12:25.941 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: size 128 MiB in 32 objects 2020-04-10T07:12:25.941 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: order 22 (4 MiB objects) 2020-04-10T07:12:25.941 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: snapshot_count: 5 2020-04-10T07:12:25.941 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: id: 11439010b41 2020-04-10T07:12:25.941 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: block_name_prefix: rbd_data.11439010b41 2020-04-10T07:12:25.941 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: format: 2 2020-04-10T07:12:25.941 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: features: layering, exclusive-lock, journaling, non-primary 2020-04-10T07:12:25.942 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: op_features: 2020-04-10T07:12:25.942 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: flags: 2020-04-10T07:12:25.942 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: create_timestamp: Fri Apr 10 07:11:09 2020 2020-04-10T07:12:25.942 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: access_timestamp: Fri Apr 10 07:11:09 2020 2020-04-10T07:12:25.942 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: modify_timestamp: Fri Apr 10 07:11:09 2020 2020-04-10T07:12:25.942 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: journal: 11439010b41 2020-04-10T07:12:25.942 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring state: enabled 2020-04-10T07:12:25.943 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring mode: snapshot 2020-04-10T07:12:25.943 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring global id: b583e294-8b69-4e57-9966-783595c454d6 2020-04-10T07:12:25.943 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring primary: false 2020-04-10T07:12:25.946 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:25.946 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test2 journal status 2020-04-10T07:12:25.946 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:25.946 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test2 journal status 2020-04-10T07:12:25.946 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror --namespace journal status --image test2 2020-04-10T07:12:26.019 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:minimum_set: 0 2020-04-10T07:12:26.019 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:active_set: 0 2020-04-10T07:12:26.019 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:registered clients: 2020-04-10T07:12:26.019 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: [id=, commit_position=[positions=[]], state=connected] 2020-04-10T07:12:26.023 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:26.023 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test_clone info 2020-04-10T07:12:26.023 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:26.024 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test_clone info 2020-04-10T07:12:26.024 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror --namespace info test_clone 2020-04-10T07:12:26.110 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:rbd image 'test_clone': 2020-04-10T07:12:26.110 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: size 128 MiB in 32 objects 2020-04-10T07:12:26.110 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: order 22 (4 MiB objects) 2020-04-10T07:12:26.110 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: snapshot_count: 2 2020-04-10T07:12:26.110 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: id: 113f209cc429 2020-04-10T07:12:26.111 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: block_name_prefix: rbd_data.113f209cc429 2020-04-10T07:12:26.111 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: format: 2 2020-04-10T07:12:26.111 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: features: layering, exclusive-lock, journaling, non-primary 2020-04-10T07:12:26.111 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: op_features: 2020-04-10T07:12:26.111 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: flags: 2020-04-10T07:12:26.111 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: create_timestamp: Fri Apr 10 07:09:49 2020 2020-04-10T07:12:26.111 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: access_timestamp: Fri Apr 10 07:09:49 2020 2020-04-10T07:12:26.112 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: modify_timestamp: Fri Apr 10 07:09:49 2020 2020-04-10T07:12:26.112 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: parent: mirror_parent/test_parent@snap 2020-04-10T07:12:26.112 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: overlap: 128 MiB 2020-04-10T07:12:26.112 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: journal: 113f209cc429 2020-04-10T07:12:26.113 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring state: enabled 2020-04-10T07:12:26.113 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring mode: snapshot 2020-04-10T07:12:26.113 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring global id: 003078e2-ed6c-4f36-8bc5-ce0c44ef25d5 2020-04-10T07:12:26.113 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring primary: false 2020-04-10T07:12:26.117 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:26.117 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test_clone journal status 2020-04-10T07:12:26.117 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:26.118 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test_clone journal status 2020-04-10T07:12:26.118 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror --namespace journal status --image test_clone 2020-04-10T07:12:26.208 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:minimum_set: 0 2020-04-10T07:12:26.209 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:active_set: 0 2020-04-10T07:12:26.209 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:registered clients: 2020-04-10T07:12:26.209 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: [id=, commit_position=[positions=[]], state=connected] 2020-04-10T07:12:26.210 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:26.211 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test_clone1 info 2020-04-10T07:12:26.211 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror --namespace info test_clone1 2020-04-10T07:12:26.211 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:26.211 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test_clone1 info 2020-04-10T07:12:26.492 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:rbd image 'test_clone1': 2020-04-10T07:12:26.493 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: size 128 MiB in 32 objects 2020-04-10T07:12:26.493 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: order 22 (4 MiB objects) 2020-04-10T07:12:26.493 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: snapshot_count: 1 2020-04-10T07:12:26.493 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: id: 134fd0e99700 2020-04-10T07:12:26.493 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: block_name_prefix: rbd_data.134fd0e99700 2020-04-10T07:12:26.493 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: format: 2 2020-04-10T07:12:26.493 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: features: layering, exclusive-lock, journaling 2020-04-10T07:12:26.494 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: op_features: 2020-04-10T07:12:26.494 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: flags: 2020-04-10T07:12:26.494 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: create_timestamp: Fri Apr 10 07:09:57 2020 2020-04-10T07:12:26.494 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: access_timestamp: Fri Apr 10 07:09:57 2020 2020-04-10T07:12:26.494 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: modify_timestamp: Fri Apr 10 07:09:57 2020 2020-04-10T07:12:26.494 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: parent: mirror_parent/test_parent@snap 2020-04-10T07:12:26.495 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: overlap: 128 MiB 2020-04-10T07:12:26.495 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: journal: 134fd0e99700 2020-04-10T07:12:26.495 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring state: enabled 2020-04-10T07:12:26.495 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring mode: snapshot 2020-04-10T07:12:26.495 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring global id: 4b1961cb-5406-443f-90f1-3627ac4ffffd 2020-04-10T07:12:26.495 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring primary: true 2020-04-10T07:12:26.499 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:26.499 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test_clone1 journal status 2020-04-10T07:12:26.500 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:26.500 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test_clone1 journal status 2020-04-10T07:12:26.500 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror --namespace journal status --image test_clone1 2020-04-10T07:12:26.601 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:minimum_set: 0 2020-04-10T07:12:26.601 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:active_set: 0 2020-04-10T07:12:26.601 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:registered clients: 2020-04-10T07:12:26.601 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: [id=, commit_position=[positions=[[object_number=1, tag_tid=1, entry_tid=1], [object_number=0, tag_tid=1, entry_tid=0]]], state=connected] 2020-04-10T07:12:26.605 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:26.605 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test_clone_v1 info 2020-04-10T07:12:26.606 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:26.606 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test_clone_v1 info 2020-04-10T07:12:26.606 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror --namespace info test_clone_v1 2020-04-10T07:12:26.696 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:rbd image 'test_clone_v1': 2020-04-10T07:12:26.696 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: size 128 MiB in 32 objects 2020-04-10T07:12:26.696 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: order 22 (4 MiB objects) 2020-04-10T07:12:26.696 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: snapshot_count: 1 2020-04-10T07:12:26.697 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: id: 114bb4bac5c 2020-04-10T07:12:26.697 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: block_name_prefix: rbd_data.114bb4bac5c 2020-04-10T07:12:26.697 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: format: 2 2020-04-10T07:12:26.697 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: features: layering, exclusive-lock, journaling, non-primary 2020-04-10T07:12:26.697 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: op_features: 2020-04-10T07:12:26.697 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: flags: 2020-04-10T07:12:26.697 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: create_timestamp: Fri Apr 10 07:10:00 2020 2020-04-10T07:12:26.698 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: access_timestamp: Fri Apr 10 07:10:00 2020 2020-04-10T07:12:26.698 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: modify_timestamp: Fri Apr 10 07:10:00 2020 2020-04-10T07:12:26.698 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: parent: mirror_parent/test_parent@snap 2020-04-10T07:12:26.698 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: overlap: 128 MiB 2020-04-10T07:12:26.698 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: journal: 114bb4bac5c 2020-04-10T07:12:26.698 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring state: enabled 2020-04-10T07:12:26.699 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring mode: snapshot 2020-04-10T07:12:26.699 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring global id: 70d86755-f73a-4bbe-9493-1b79a191cc0c 2020-04-10T07:12:26.699 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring primary: false 2020-04-10T07:12:26.703 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:26.704 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test_clone_v1 journal status 2020-04-10T07:12:26.704 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:26.704 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test_clone_v1 journal status 2020-04-10T07:12:26.704 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror --namespace journal status --image test_clone_v1 2020-04-10T07:12:26.803 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:minimum_set: 0 2020-04-10T07:12:26.804 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:active_set: 0 2020-04-10T07:12:26.804 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:registered clients: 2020-04-10T07:12:26.804 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: [id=, commit_position=[positions=[]], state=connected] 2020-04-10T07:12:26.808 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:26.808 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test_data_pool info 2020-04-10T07:12:26.809 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:26.809 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test_data_pool info 2020-04-10T07:12:26.809 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror --namespace info test_data_pool 2020-04-10T07:12:26.877 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:rbd image 'test_data_pool': 2020-04-10T07:12:26.877 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: size 128 MiB in 32 objects 2020-04-10T07:12:26.877 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: order 22 (4 MiB objects) 2020-04-10T07:12:26.877 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: snapshot_count: 4 2020-04-10T07:12:26.878 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: id: 113fde7aeaa 2020-04-10T07:12:26.878 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: data_pool: mirror_parent 2020-04-10T07:12:26.878 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: block_name_prefix: rbd_data.3.113fde7aeaa 2020-04-10T07:12:26.878 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: format: 2 2020-04-10T07:12:26.878 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: features: layering, exclusive-lock, journaling, data-pool, non-primary 2020-04-10T07:12:26.878 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: op_features: 2020-04-10T07:12:26.879 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: flags: 2020-04-10T07:12:26.879 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: create_timestamp: Fri Apr 10 07:10:23 2020 2020-04-10T07:12:26.879 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: access_timestamp: Fri Apr 10 07:10:23 2020 2020-04-10T07:12:26.879 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: modify_timestamp: Fri Apr 10 07:10:23 2020 2020-04-10T07:12:26.879 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: journal: 113fde7aeaa 2020-04-10T07:12:26.879 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring state: enabled 2020-04-10T07:12:26.879 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring mode: snapshot 2020-04-10T07:12:26.880 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring global id: a8183526-4331-4f0d-af4a-7c93c501e927 2020-04-10T07:12:26.880 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring primary: false 2020-04-10T07:12:26.884 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:26.884 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test_data_pool journal status 2020-04-10T07:12:26.884 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:26.884 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test_data_pool journal status 2020-04-10T07:12:26.884 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror --namespace journal status --image test_data_pool 2020-04-10T07:12:26.958 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:minimum_set: 0 2020-04-10T07:12:26.959 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:active_set: 0 2020-04-10T07:12:26.959 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:registered clients: 2020-04-10T07:12:26.959 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: [id=, commit_position=[positions=[]], state=connected] 2020-04-10T07:12:26.963 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:26.963 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test_force_promote info 2020-04-10T07:12:26.963 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror --namespace info test_force_promote 2020-04-10T07:12:26.964 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:26.964 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test_force_promote info 2020-04-10T07:12:27.027 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:rbd image 'test_force_promote': 2020-04-10T07:12:27.027 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: size 128 MiB in 32 objects 2020-04-10T07:12:27.028 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: order 22 (4 MiB objects) 2020-04-10T07:12:27.028 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: snapshot_count: 4 2020-04-10T07:12:27.028 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: id: 115186996acb 2020-04-10T07:12:27.028 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: block_name_prefix: rbd_data.115186996acb 2020-04-10T07:12:27.028 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: format: 2 2020-04-10T07:12:27.028 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: features: layering, exclusive-lock, journaling 2020-04-10T07:12:27.029 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: op_features: 2020-04-10T07:12:27.029 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: flags: 2020-04-10T07:12:27.029 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: create_timestamp: Fri Apr 10 07:09:23 2020 2020-04-10T07:12:27.029 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: access_timestamp: Fri Apr 10 07:09:23 2020 2020-04-10T07:12:27.029 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: modify_timestamp: Fri Apr 10 07:09:23 2020 2020-04-10T07:12:27.029 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: journal: 115186996acb 2020-04-10T07:12:27.029 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring state: enabled 2020-04-10T07:12:27.029 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring mode: snapshot 2020-04-10T07:12:27.030 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring global id: b192e243-6abc-405a-b798-b60b9d74691e 2020-04-10T07:12:27.030 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring primary: true 2020-04-10T07:12:27.032 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.033 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test_force_promote journal status 2020-04-10T07:12:27.033 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:27.033 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test_force_promote journal status 2020-04-10T07:12:27.033 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror --namespace journal status --image test_force_promote 2020-04-10T07:12:27.110 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:minimum_set: 0 2020-04-10T07:12:27.110 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:active_set: 0 2020-04-10T07:12:27.110 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:registered clients: 2020-04-10T07:12:27.110 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: [id=, commit_position=[positions=[[object_number=1, tag_tid=2, entry_tid=101], [object_number=0, tag_tid=2, entry_tid=100], [object_number=3, tag_tid=2, entry_tid=99], [object_number=2, tag_tid=2, entry_tid=98]]], state=connected] 2020-04-10T07:12:27.114 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:27.114 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1 mirror rbd_mirroring omap vals 2020-04-10T07:12:27.115 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rados --cluster cluster1 -p mirror --namespace listomapvals rbd_mirroring 2020-04-10T07:12:27.115 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.115 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster1 mirror rbd_mirroring omap vals 2020-04-10T07:12:27.160 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:global_003078e2-ed6c-4f36-8bc5-ce0c44ef25d5 2020-04-10T07:12:27.160 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (16 bytes) : 2020-04-10T07:12:27.161 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 0c 00 00 00 31 31 33 66 32 30 39 63 63 34 32 39 |....113f209cc429| 2020-04-10T07:12:27.161 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 2020-04-10T07:12:27.161 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.161 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:global_4b1961cb-5406-443f-90f1-3627ac4ffffd 2020-04-10T07:12:27.161 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (16 bytes) : 2020-04-10T07:12:27.161 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 0c 00 00 00 31 33 34 66 64 30 65 39 39 37 30 30 |....134fd0e99700| 2020-04-10T07:12:27.161 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 2020-04-10T07:12:27.162 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.162 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:global_56419286-13e9-49f6-a20d-2abe6ede083e 2020-04-10T07:12:27.162 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (16 bytes) : 2020-04-10T07:12:27.162 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 0c 00 00 00 31 31 33 66 35 61 65 64 30 33 61 30 |....113f5aed03a0| 2020-04-10T07:12:27.162 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 2020-04-10T07:12:27.162 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.162 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:global_70d86755-f73a-4bbe-9493-1b79a191cc0c 2020-04-10T07:12:27.163 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (15 bytes) : 2020-04-10T07:12:27.163 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 0b 00 00 00 31 31 34 62 62 34 62 61 63 35 63 |....114bb4bac5c| 2020-04-10T07:12:27.163 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000000f 2020-04-10T07:12:27.163 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.163 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:global_a8183526-4331-4f0d-af4a-7c93c501e927 2020-04-10T07:12:27.163 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (15 bytes) : 2020-04-10T07:12:27.163 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 0b 00 00 00 31 31 33 66 64 65 37 61 65 61 61 |....113fde7aeaa| 2020-04-10T07:12:27.163 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000000f 2020-04-10T07:12:27.164 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.164 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:global_a9af7d9e-1404-4730-9e18-c0418a915346 2020-04-10T07:12:27.164 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (16 bytes) : 2020-04-10T07:12:27.164 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 0c 00 00 00 31 31 34 62 63 65 65 31 37 39 64 61 |....114bcee179da| 2020-04-10T07:12:27.164 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 2020-04-10T07:12:27.164 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.165 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:global_b192e243-6abc-405a-b798-b60b9d74691e 2020-04-10T07:12:27.165 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (16 bytes) : 2020-04-10T07:12:27.166 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 0c 00 00 00 31 31 35 31 38 36 39 39 36 61 63 62 |....115186996acb| 2020-04-10T07:12:27.166 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 2020-04-10T07:12:27.166 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.166 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:global_b583e294-8b69-4e57-9966-783595c454d6 2020-04-10T07:12:27.166 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (15 bytes) : 2020-04-10T07:12:27.166 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 0b 00 00 00 31 31 34 33 39 30 31 30 62 34 31 |....11439010b41| 2020-04-10T07:12:27.166 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000000f 2020-04-10T07:12:27.167 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.167 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image_113f209cc429 2020-04-10T07:12:27.167 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (48 bytes) : 2020-04-10T07:12:27.167 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 02 01 2a 00 00 00 24 00 00 00 30 30 33 30 37 38 |..*...$...003078| 2020-04-10T07:12:27.167 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 65 32 2d 65 64 36 63 2d 34 66 33 36 2d 38 62 63 |e2-ed6c-4f36-8bc| 2020-04-10T07:12:27.167 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 35 2d 63 65 30 63 34 34 65 66 32 35 64 35 01 01 |5-ce0c44ef25d5..| 2020-04-10T07:12:27.167 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 2020-04-10T07:12:27.168 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.168 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image_113f5aed03a0 2020-04-10T07:12:27.168 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (48 bytes) : 2020-04-10T07:12:27.169 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 02 01 2a 00 00 00 24 00 00 00 35 36 34 31 39 32 |..*...$...564192| 2020-04-10T07:12:27.169 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 38 36 2d 31 33 65 39 2d 34 39 66 36 2d 61 32 30 |86-13e9-49f6-a20| 2020-04-10T07:12:27.170 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 64 2d 32 61 62 65 36 65 64 65 30 38 33 65 01 01 |d-2abe6ede083e..| 2020-04-10T07:12:27.170 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 2020-04-10T07:12:27.170 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.170 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image_113fde7aeaa 2020-04-10T07:12:27.170 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (48 bytes) : 2020-04-10T07:12:27.170 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 02 01 2a 00 00 00 24 00 00 00 61 38 31 38 33 35 |..*...$...a81835| 2020-04-10T07:12:27.170 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 32 36 2d 34 33 33 31 2d 34 66 30 64 2d 61 66 34 |26-4331-4f0d-af4| 2020-04-10T07:12:27.171 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 61 2d 37 63 39 33 63 35 30 31 65 39 32 37 01 01 |a-7c93c501e927..| 2020-04-10T07:12:27.171 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 2020-04-10T07:12:27.171 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.171 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image_11439010b41 2020-04-10T07:12:27.171 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (48 bytes) : 2020-04-10T07:12:27.171 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 02 01 2a 00 00 00 24 00 00 00 62 35 38 33 65 32 |..*...$...b583e2| 2020-04-10T07:12:27.171 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 39 34 2d 38 62 36 39 2d 34 65 35 37 2d 39 39 36 |94-8b69-4e57-996| 2020-04-10T07:12:27.171 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 36 2d 37 38 33 35 39 35 63 34 35 34 64 36 01 01 |6-783595c454d6..| 2020-04-10T07:12:27.172 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 2020-04-10T07:12:27.172 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.172 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image_114bb4bac5c 2020-04-10T07:12:27.172 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (48 bytes) : 2020-04-10T07:12:27.172 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 02 01 2a 00 00 00 24 00 00 00 37 30 64 38 36 37 |..*...$...70d867| 2020-04-10T07:12:27.172 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 35 35 2d 66 37 33 61 2d 34 62 62 65 2d 39 34 39 |55-f73a-4bbe-949| 2020-04-10T07:12:27.172 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 33 2d 31 62 37 39 61 31 39 31 63 63 30 63 01 01 |3-1b79a191cc0c..| 2020-04-10T07:12:27.173 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 2020-04-10T07:12:27.173 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.173 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image_114bcee179da 2020-04-10T07:12:27.173 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (48 bytes) : 2020-04-10T07:12:27.173 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 02 01 2a 00 00 00 24 00 00 00 61 39 61 66 37 64 |..*...$...a9af7d| 2020-04-10T07:12:27.173 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 39 65 2d 31 34 30 34 2d 34 37 33 30 2d 39 65 31 |9e-1404-4730-9e1| 2020-04-10T07:12:27.173 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 38 2d 63 30 34 31 38 61 39 31 35 33 34 36 01 01 |8-c0418a915346..| 2020-04-10T07:12:27.174 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 2020-04-10T07:12:27.174 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.174 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image_115186996acb 2020-04-10T07:12:27.174 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (48 bytes) : 2020-04-10T07:12:27.174 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 02 01 2a 00 00 00 24 00 00 00 62 31 39 32 65 32 |..*...$...b192e2| 2020-04-10T07:12:27.174 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 34 33 2d 36 61 62 63 2d 34 30 35 61 2d 62 37 39 |43-6abc-405a-b79| 2020-04-10T07:12:27.174 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 38 2d 62 36 30 62 39 64 37 34 36 39 31 65 01 01 |8-b60b9d74691e..| 2020-04-10T07:12:27.174 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 2020-04-10T07:12:27.175 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.175 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image_134fd0e99700 2020-04-10T07:12:27.175 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (48 bytes) : 2020-04-10T07:12:27.175 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 02 01 2a 00 00 00 24 00 00 00 34 62 31 39 36 31 |..*...$...4b1961| 2020-04-10T07:12:27.175 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 63 62 2d 35 34 30 36 2d 34 34 33 66 2d 39 30 66 |cb-5406-443f-90f| 2020-04-10T07:12:27.175 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 31 2d 33 36 32 37 61 63 34 66 66 66 66 64 01 01 |1-3627ac4ffffd..| 2020-04-10T07:12:27.175 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 2020-04-10T07:12:27.176 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.176 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:mirror_mode 2020-04-10T07:12:27.176 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (4 bytes) : 2020-04-10T07:12:27.176 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 00 00 00 |....| 2020-04-10T07:12:27.176 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000004 2020-04-10T07:12:27.176 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.176 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:mirror_peer_2571f49c-d767-4112-ae12-d8b71263d58b 2020-04-10T07:12:27.178 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (136 bytes) : 2020-04-10T07:12:27.178 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 02 01 82 00 00 00 24 00 00 00 32 35 37 31 66 34 |......$...2571f4| 2020-04-10T07:12:27.179 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 39 63 2d 64 37 36 37 2d 34 31 31 32 2d 61 65 31 |9c-d767-4112-ae1| 2020-04-10T07:12:27.179 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 32 2d 64 38 62 37 31 32 36 33 64 35 38 62 0c 00 |2-d8b71263d58b..| 2020-04-10T07:12:27.179 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 63 6c 75 73 74 65 72 32 2d 44 4e 45 0d 00 |..cluster2-DNE..| 2020-04-10T07:12:27.179 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 00 00 63 6c 69 65 6e 74 2e 6d 69 72 72 6f 72 ff |..client.mirror.| 2020-04-10T07:12:27.179 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 ff ff ff ff ff ff ff 02 24 00 00 00 38 32 35 33 |........$...8253| 2020-04-10T07:12:27.179 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000060 66 64 65 32 2d 38 36 66 38 2d 34 34 33 61 2d 62 |fde2-86f8-443a-b| 2020-04-10T07:12:27.179 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000070 34 62 66 2d 65 63 37 64 61 65 32 66 61 66 61 33 |4bf-ec7dae2fafa3| 2020-04-10T07:12:27.180 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000080 46 1c 90 5e de db fb 0b |F..^....| 2020-04-10T07:12:27.180 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000088 2020-04-10T07:12:27.180 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.180 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:mirror_uuid 2020-04-10T07:12:27.180 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (36 bytes) : 2020-04-10T07:12:27.180 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 36 32 30 64 64 62 30 30 2d 30 66 34 39 2d 34 37 |620ddb00-0f49-47| 2020-04-10T07:12:27.180 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 35 38 2d 61 32 31 36 2d 38 64 39 63 63 38 39 30 |58-a216-8d9cc890| 2020-04-10T07:12:27.181 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 64 30 66 61 |d0fa| 2020-04-10T07:12:27.181 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000024 2020-04-10T07:12:27.181 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.181 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:remote_status_global_003078e2-ed6c-4f36-8bc5-ce0c44ef25d5_8253fde2-86f8-443a-b4bf-ec7dae2fafa3 2020-04-10T07:12:27.181 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (92 bytes) : 2020-04-10T07:12:27.181 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 4e 12 00 00 00 00 00 00 01 |..,....N........| 2020-04-10T07:12:27.181 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 aa d9 82 52 10 00 |.............R..| 2020-04-10T07:12:27.182 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T07:12:27.182 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 24 00 00 00 06 16 00 00 00 6c 6f 63 |....$........loc| 2020-04-10T07:12:27.182 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 61 6c 20 69 6d 61 67 65 20 69 73 20 70 72 69 6d |al image is prim| 2020-04-10T07:12:27.182 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 61 72 79 5a 1c 90 5e 2a 44 a0 0b 00 |aryZ..^*D...| 2020-04-10T07:12:27.182 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000005c 2020-04-10T07:12:27.182 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.183 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:remote_status_global_3876de2e-b17d-4c86-94c4-90a11916f44e_8253fde2-86f8-443a-b4bf-ec7dae2fafa3 2020-04-10T07:12:27.183 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (96 bytes) : 2020-04-10T07:12:27.183 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 4e 12 00 00 00 00 00 00 01 |..,....N........| 2020-04-10T07:12:27.183 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 aa d9 82 52 10 00 |.............R..| 2020-04-10T07:12:27.183 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T07:12:27.183 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 28 00 00 00 01 1a 00 00 00 65 72 72 |....(........err| 2020-04-10T07:12:27.183 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 6f 72 20 62 6f 6f 74 73 74 72 61 70 70 69 6e 67 |or bootstrapping| 2020-04-10T07:12:27.183 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 20 72 65 70 6c 61 79 39 1c 90 5e 0c 9d b3 03 00 | replay9..^.....| 2020-04-10T07:12:27.184 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000060 2020-04-10T07:12:27.184 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.184 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:remote_status_global_4334771a-4312-4fab-8e12-36d78acc7af1_8253fde2-86f8-443a-b4bf-ec7dae2fafa3 2020-04-10T07:12:27.184 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (96 bytes) : 2020-04-10T07:12:27.184 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 4c 12 00 00 00 00 00 00 01 |..,....L........| 2020-04-10T07:12:27.184 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 a5 ab 32 c8 10 00 |............2...| 2020-04-10T07:12:27.185 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T07:12:27.185 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 28 00 00 00 01 1a 00 00 00 65 72 72 |....(........err| 2020-04-10T07:12:27.185 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 6f 72 20 62 6f 6f 74 73 74 72 61 70 70 69 6e 67 |or bootstrapping| 2020-04-10T07:12:27.186 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 20 72 65 70 6c 61 79 18 1c 90 5e e2 03 d0 39 00 | replay...^...9.| 2020-04-10T07:12:27.186 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000060 2020-04-10T07:12:27.186 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.186 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:remote_status_global_4b1961cb-5406-443f-90f1-3627ac4ffffd_8253fde2-86f8-443a-b4bf-ec7dae2fafa3 2020-04-10T07:12:27.186 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (186 bytes) : 2020-04-10T07:12:27.186 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 4c 12 00 00 00 00 00 00 01 |..,....L........| 2020-04-10T07:12:27.186 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 a5 ab 32 c8 10 00 |............2...| 2020-04-10T07:12:27.187 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T07:12:27.187 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 82 00 00 00 04 74 00 00 00 72 65 70 |.........t...rep| 2020-04-10T07:12:27.187 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 6c 61 79 69 6e 67 2c 20 7b 22 72 65 70 6c 61 79 |laying, {"replay| 2020-04-10T07:12:27.187 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 5f 73 74 61 74 65 22 3a 20 22 69 64 6c 65 22 2c |_state": "idle",| 2020-04-10T07:12:27.187 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000060 20 22 72 65 6d 6f 74 65 5f 73 6e 61 70 73 68 6f | "remote_snapsho| 2020-04-10T07:12:27.187 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000070 74 5f 74 69 6d 65 73 74 61 6d 70 22 3a 20 31 35 |t_timestamp": 15| 2020-04-10T07:12:27.187 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000080 38 36 35 30 32 35 39 38 2c 20 22 6c 6f 63 61 6c |86502598, "local| 2020-04-10T07:12:27.187 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000090 5f 73 6e 61 70 73 68 6f 74 5f 74 69 6d 65 73 74 |_snapshot_timest| 2020-04-10T07:12:27.188 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:000000a0 61 6d 70 22 3a 20 31 35 38 36 35 30 32 35 39 38 |amp": 1586502598| 2020-04-10T07:12:27.188 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:000000b0 7d 46 1c 90 5e 28 13 8d 09 00 |}F..^(....| 2020-04-10T07:12:27.188 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:000000ba 2020-04-10T07:12:27.188 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.188 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:remote_status_global_56419286-13e9-49f6-a20d-2abe6ede083e_8253fde2-86f8-443a-b4bf-ec7dae2fafa3 2020-04-10T07:12:27.188 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (92 bytes) : 2020-04-10T07:12:27.188 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 4e 12 00 00 00 00 00 00 01 |..,....N........| 2020-04-10T07:12:27.189 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 aa d9 82 52 10 00 |.............R..| 2020-04-10T07:12:27.189 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T07:12:27.189 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 24 00 00 00 06 16 00 00 00 6c 6f 63 |....$........loc| 2020-04-10T07:12:27.189 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 61 6c 20 69 6d 61 67 65 20 69 73 20 70 72 69 6d |al image is prim| 2020-04-10T07:12:27.189 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 61 72 79 5a 1c 90 5e 9b 7e c8 0b 00 |aryZ..^.~...| 2020-04-10T07:12:27.189 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000005c 2020-04-10T07:12:27.189 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.190 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:remote_status_global_5d3e94e2-1c6c-4146-b461-67ecadf338c4_8253fde2-86f8-443a-b4bf-ec7dae2fafa3 2020-04-10T07:12:27.190 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (96 bytes) : 2020-04-10T07:12:27.190 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 4c 12 00 00 00 00 00 00 01 |..,....L........| 2020-04-10T07:12:27.190 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 a5 ab 32 c8 10 00 |............2...| 2020-04-10T07:12:27.190 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T07:12:27.190 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 28 00 00 00 01 1a 00 00 00 65 72 72 |....(........err| 2020-04-10T07:12:27.191 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 6f 72 20 62 6f 6f 74 73 74 72 61 70 70 69 6e 67 |or bootstrapping| 2020-04-10T07:12:27.191 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 20 72 65 70 6c 61 79 18 1c 90 5e 32 84 d2 39 00 | replay...^2..9.| 2020-04-10T07:12:27.191 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000060 2020-04-10T07:12:27.191 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.191 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:remote_status_global_697ea219-b252-4c33-ba20-9b31ea5be82a_8253fde2-86f8-443a-b4bf-ec7dae2fafa3 2020-04-10T07:12:27.191 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (96 bytes) : 2020-04-10T07:12:27.191 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 4e 12 00 00 00 00 00 00 01 |..,....N........| 2020-04-10T07:12:27.191 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 aa d9 82 52 10 00 |.............R..| 2020-04-10T07:12:27.192 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T07:12:27.192 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 28 00 00 00 01 1a 00 00 00 65 72 72 |....(........err| 2020-04-10T07:12:27.192 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 6f 72 20 62 6f 6f 74 73 74 72 61 70 70 69 6e 67 |or bootstrapping| 2020-04-10T07:12:27.192 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 20 72 65 70 6c 61 79 18 1c 90 5e 6d bf 2d 3a 00 | replay...^m.-:.| 2020-04-10T07:12:27.194 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000060 2020-04-10T07:12:27.194 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.194 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:remote_status_global_70d86755-f73a-4bbe-9493-1b79a191cc0c_8253fde2-86f8-443a-b4bf-ec7dae2fafa3 2020-04-10T07:12:27.195 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (92 bytes) : 2020-04-10T07:12:27.195 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 4c 12 00 00 00 00 00 00 01 |..,....L........| 2020-04-10T07:12:27.195 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 a5 ab 32 c8 10 00 |............2...| 2020-04-10T07:12:27.195 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T07:12:27.195 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 24 00 00 00 06 16 00 00 00 6c 6f 63 |....$........loc| 2020-04-10T07:12:27.195 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 61 6c 20 69 6d 61 67 65 20 69 73 20 70 72 69 6d |al image is prim| 2020-04-10T07:12:27.195 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 61 72 79 5a 1c 90 5e 97 8e d8 0a 00 |aryZ..^.....| 2020-04-10T07:12:27.196 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000005c 2020-04-10T07:12:27.196 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.196 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:remote_status_global_79225ce2-9203-4752-bc9b-35a11186acdb_8253fde2-86f8-443a-b4bf-ec7dae2fafa3 2020-04-10T07:12:27.196 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (96 bytes) : 2020-04-10T07:12:27.196 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 4e 12 00 00 00 00 00 00 01 |..,....N........| 2020-04-10T07:12:27.196 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 aa d9 82 52 10 00 |.............R..| 2020-04-10T07:12:27.197 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T07:12:27.197 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 28 00 00 00 01 1a 00 00 00 65 72 72 |....(........err| 2020-04-10T07:12:27.198 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 6f 72 20 62 6f 6f 74 73 74 72 61 70 70 69 6e 67 |or bootstrapping| 2020-04-10T07:12:27.198 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 20 72 65 70 6c 61 79 5a 1c 90 5e 24 08 12 0b 00 | replayZ..^$....| 2020-04-10T07:12:27.198 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000060 2020-04-10T07:12:27.198 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.198 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:remote_status_global_925c4d02-9086-4edd-b4f3-79c1b7e3734c_8253fde2-86f8-443a-b4bf-ec7dae2fafa3 2020-04-10T07:12:27.198 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (96 bytes) : 2020-04-10T07:12:27.198 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 4c 12 00 00 00 00 00 00 01 |..,....L........| 2020-04-10T07:12:27.199 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 a5 ab 32 c8 10 00 |............2...| 2020-04-10T07:12:27.199 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T07:12:27.199 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 28 00 00 00 01 1a 00 00 00 65 72 72 |....(........err| 2020-04-10T07:12:27.199 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 6f 72 20 62 6f 6f 74 73 74 72 61 70 70 69 6e 67 |or bootstrapping| 2020-04-10T07:12:27.199 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 20 72 65 70 6c 61 79 39 1c 90 5e 75 0e fd 03 00 | replay9..^u....| 2020-04-10T07:12:27.199 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000060 2020-04-10T07:12:27.199 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.200 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:remote_status_global_a8183526-4331-4f0d-af4a-7c93c501e927_8253fde2-86f8-443a-b4bf-ec7dae2fafa3 2020-04-10T07:12:27.200 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (92 bytes) : 2020-04-10T07:12:27.200 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 4e 12 00 00 00 00 00 00 01 |..,....N........| 2020-04-10T07:12:27.200 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 aa d9 82 52 10 00 |.............R..| 2020-04-10T07:12:27.200 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T07:12:27.200 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 24 00 00 00 06 16 00 00 00 6c 6f 63 |....$........loc| 2020-04-10T07:12:27.200 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 61 6c 20 69 6d 61 67 65 20 69 73 20 70 72 69 6d |al image is prim| 2020-04-10T07:12:27.200 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 61 72 79 5a 1c 90 5e 56 fc c8 0b 00 |aryZ..^V....| 2020-04-10T07:12:27.201 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000005c 2020-04-10T07:12:27.201 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.201 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:remote_status_global_a9af7d9e-1404-4730-9e18-c0418a915346_8253fde2-86f8-443a-b4bf-ec7dae2fafa3 2020-04-10T07:12:27.201 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (92 bytes) : 2020-04-10T07:12:27.201 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 4e 12 00 00 00 00 00 00 01 |..,....N........| 2020-04-10T07:12:27.201 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 aa d9 82 52 10 00 |.............R..| 2020-04-10T07:12:27.201 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T07:12:27.202 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 24 00 00 00 06 16 00 00 00 6c 6f 63 |....$........loc| 2020-04-10T07:12:27.202 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 61 6c 20 69 6d 61 67 65 20 69 73 20 70 72 69 6d |al image is prim| 2020-04-10T07:12:27.202 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 61 72 79 5a 1c 90 5e 4e 91 f0 0b 00 |aryZ..^N....| 2020-04-10T07:12:27.202 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000005c 2020-04-10T07:12:27.202 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.202 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:remote_status_global_b192e243-6abc-405a-b798-b60b9d74691e_8253fde2-86f8-443a-b4bf-ec7dae2fafa3 2020-04-10T07:12:27.202 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (92 bytes) : 2020-04-10T07:12:27.202 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 4c 12 00 00 00 00 00 00 01 |..,....L........| 2020-04-10T07:12:27.203 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 a5 ab 32 c8 10 00 |............2...| 2020-04-10T07:12:27.203 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T07:12:27.203 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 24 00 00 00 06 16 00 00 00 6c 6f 63 |....$........loc| 2020-04-10T07:12:27.203 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 61 6c 20 69 6d 61 67 65 20 69 73 20 70 72 69 6d |al image is prim| 2020-04-10T07:12:27.203 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 61 72 79 5a 1c 90 5e 37 72 3b 0b 00 |aryZ..^7r;..| 2020-04-10T07:12:27.203 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000005c 2020-04-10T07:12:27.203 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.204 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:remote_status_global_b583e294-8b69-4e57-9966-783595c454d6_8253fde2-86f8-443a-b4bf-ec7dae2fafa3 2020-04-10T07:12:27.204 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (92 bytes) : 2020-04-10T07:12:27.204 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 4c 12 00 00 00 00 00 00 01 |..,....L........| 2020-04-10T07:12:27.204 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 a5 ab 32 c8 10 00 |............2...| 2020-04-10T07:12:27.204 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T07:12:27.204 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 24 00 00 00 06 16 00 00 00 6c 6f 63 |....$........loc| 2020-04-10T07:12:27.204 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 61 6c 20 69 6d 61 67 65 20 69 73 20 70 72 69 6d |al image is prim| 2020-04-10T07:12:27.205 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 61 72 79 5a 1c 90 5e 72 40 72 0b 00 |aryZ..^r@r..| 2020-04-10T07:12:27.205 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000005c 2020-04-10T07:12:27.205 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.205 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:status_global_003078e2-ed6c-4f36-8bc5-ce0c44ef25d5 2020-04-10T07:12:27.205 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (77 bytes) : 2020-04-10T07:12:27.205 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 3f 11 00 00 00 00 00 00 01 |..,....?........| 2020-04-10T07:12:27.205 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 95 c4 de e4 10 00 |................| 2020-04-10T07:12:27.206 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T07:12:27.206 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 15 00 00 00 06 07 00 00 00 73 74 6f |.............sto| 2020-04-10T07:12:27.206 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 70 70 65 64 12 1c 90 5e 8b 60 e6 17 00 |pped...^.`...| 2020-04-10T07:12:27.206 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000004d 2020-04-10T07:12:27.206 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.206 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:status_global_3876de2e-b17d-4c86-94c4-90a11916f44e 2020-04-10T07:12:27.207 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (96 bytes) : 2020-04-10T07:12:27.207 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 4b 11 00 00 00 00 00 00 01 |..,....K........| 2020-04-10T07:12:27.207 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 c1 9f 21 4e 10 00 |............!N..| 2020-04-10T07:12:27.207 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T07:12:27.207 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 28 00 00 00 01 1a 00 00 00 65 72 72 |....(........err| 2020-04-10T07:12:27.208 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 6f 72 20 62 6f 6f 74 73 74 72 61 70 70 69 6e 67 |or bootstrapping| 2020-04-10T07:12:27.208 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 20 72 65 70 6c 61 79 14 1c 90 5e 31 83 23 21 00 | replay...^1.#!.| 2020-04-10T07:12:27.208 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000060 2020-04-10T07:12:27.208 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.208 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:status_global_4334771a-4312-4fab-8e12-36d78acc7af1 2020-04-10T07:12:27.208 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (96 bytes) : 2020-04-10T07:12:27.209 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 51 11 00 00 00 00 00 00 01 |..,....Q........| 2020-04-10T07:12:27.209 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 60 5f b6 16 10 00 |..........`_....| 2020-04-10T07:12:27.209 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T07:12:27.209 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 28 00 00 00 01 1a 00 00 00 65 72 72 |....(........err| 2020-04-10T07:12:27.209 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 6f 72 20 62 6f 6f 74 73 74 72 61 70 70 69 6e 67 |or bootstrapping| 2020-04-10T07:12:27.209 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 20 72 65 70 6c 61 79 14 1c 90 5e cd 14 5c 21 00 | replay...^..\!.| 2020-04-10T07:12:27.209 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000060 2020-04-10T07:12:27.209 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.210 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:status_global_4b1961cb-5406-443f-90f1-3627ac4ffffd 2020-04-10T07:12:27.210 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (92 bytes) : 2020-04-10T07:12:27.210 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 43 11 00 00 00 00 00 00 01 |..,....C........| 2020-04-10T07:12:27.210 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 09 d2 60 2c 10 00 |............`,..| 2020-04-10T07:12:27.210 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T07:12:27.210 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 24 00 00 00 06 16 00 00 00 6c 6f 63 |....$........loc| 2020-04-10T07:12:27.210 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 61 6c 20 69 6d 61 67 65 20 69 73 20 70 72 69 6d |al image is prim| 2020-04-10T07:12:27.211 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 61 72 79 13 1c 90 5e e4 96 14 1b 00 |ary...^.....| 2020-04-10T07:12:27.211 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000005c 2020-04-10T07:12:27.211 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.211 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:status_global_56419286-13e9-49f6-a20d-2abe6ede083e 2020-04-10T07:12:27.211 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (77 bytes) : 2020-04-10T07:12:27.211 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 3f 11 00 00 00 00 00 00 01 |..,....?........| 2020-04-10T07:12:27.211 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 95 c4 de e4 10 00 |................| 2020-04-10T07:12:27.211 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T07:12:27.212 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 15 00 00 00 06 07 00 00 00 73 74 6f |.............sto| 2020-04-10T07:12:27.212 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 70 70 65 64 12 1c 90 5e 70 41 98 17 00 |pped...^pA...| 2020-04-10T07:12:27.212 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000004d 2020-04-10T07:12:27.212 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.212 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:status_global_5d3e94e2-1c6c-4146-b461-67ecadf338c4 2020-04-10T07:12:27.212 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (96 bytes) : 2020-04-10T07:12:27.212 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 51 11 00 00 00 00 00 00 01 |..,....Q........| 2020-04-10T07:12:27.213 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 60 5f b6 16 10 00 |..........`_....| 2020-04-10T07:12:27.213 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T07:12:27.213 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 28 00 00 00 01 1a 00 00 00 65 72 72 |....(........err| 2020-04-10T07:12:27.213 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 6f 72 20 62 6f 6f 74 73 74 72 61 70 70 69 6e 67 |or bootstrapping| 2020-04-10T07:12:27.213 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 20 72 65 70 6c 61 79 14 1c 90 5e fd 32 81 21 00 | replay...^.2.!.| 2020-04-10T07:12:27.213 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000060 2020-04-10T07:12:27.213 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.214 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:status_global_687e9f7d-ae46-4324-b8f7-7c16605e0fa6 2020-04-10T07:12:27.214 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (96 bytes) : 2020-04-10T07:12:27.214 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 43 11 00 00 00 00 00 00 01 |..,....C........| 2020-04-10T07:12:27.214 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 09 d2 60 2c 10 00 |............`,..| 2020-04-10T07:12:27.214 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T07:12:27.214 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 28 00 00 00 01 1a 00 00 00 65 72 72 |....(........err| 2020-04-10T07:12:27.214 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 6f 72 20 62 6f 6f 74 73 74 72 61 70 70 69 6e 67 |or bootstrapping| 2020-04-10T07:12:27.214 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 20 72 65 70 6c 61 79 13 1c 90 5e ed 3e 4c 1a 00 | replay...^.>L..| 2020-04-10T07:12:27.215 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000060 2020-04-10T07:12:27.215 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.215 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:status_global_697ea219-b252-4c33-ba20-9b31ea5be82a 2020-04-10T07:12:27.215 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (96 bytes) : 2020-04-10T07:12:27.215 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 3f 11 00 00 00 00 00 00 01 |..,....?........| 2020-04-10T07:12:27.215 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 95 c4 de e4 10 00 |................| 2020-04-10T07:12:27.215 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T07:12:27.216 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 28 00 00 00 01 1a 00 00 00 65 72 72 |....(........err| 2020-04-10T07:12:27.216 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 6f 72 20 62 6f 6f 74 73 74 72 61 70 70 69 6e 67 |or bootstrapping| 2020-04-10T07:12:27.216 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 20 72 65 70 6c 61 79 0d 1c 90 5e 2e 8e 6b 16 00 | replay...^..k..| 2020-04-10T07:12:27.216 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000060 2020-04-10T07:12:27.216 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.216 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:status_global_70d86755-f73a-4bbe-9493-1b79a191cc0c 2020-04-10T07:12:27.216 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (77 bytes) : 2020-04-10T07:12:27.217 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 4b 11 00 00 00 00 00 00 01 |..,....K........| 2020-04-10T07:12:27.217 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 c1 9f 21 4e 10 00 |............!N..| 2020-04-10T07:12:27.217 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T07:12:27.217 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 15 00 00 00 06 07 00 00 00 73 74 6f |.............sto| 2020-04-10T07:12:27.217 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 70 70 65 64 14 1c 90 5e 3e d4 e9 26 00 |pped...^>..&.| 2020-04-10T07:12:27.217 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000004d 2020-04-10T07:12:27.217 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.217 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:status_global_79225ce2-9203-4752-bc9b-35a11186acdb 2020-04-10T07:12:27.218 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (97 bytes) : 2020-04-10T07:12:27.218 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 4b 11 00 00 00 00 00 00 01 |..,....K........| 2020-04-10T07:12:27.218 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 c1 9f 21 4e 10 00 |............!N..| 2020-04-10T07:12:27.218 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T07:12:27.218 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 29 00 00 00 00 1b 00 00 00 72 65 6d |....)........rem| 2020-04-10T07:12:27.218 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 6f 74 65 20 69 6d 61 67 65 20 69 73 20 6e 6f 6e |ote image is non| 2020-04-10T07:12:27.218 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 2d 70 72 69 6d 61 72 79 14 1c 90 5e 79 14 9d 21 |-primary...^y..!| 2020-04-10T07:12:27.219 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000060 00 |.| 2020-04-10T07:12:27.219 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000061 2020-04-10T07:12:27.219 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.219 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:status_global_925c4d02-9086-4edd-b4f3-79c1b7e3734c 2020-04-10T07:12:27.219 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (96 bytes) : 2020-04-10T07:12:27.219 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 43 11 00 00 00 00 00 00 01 |..,....C........| 2020-04-10T07:12:27.219 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 09 d2 60 2c 10 00 |............`,..| 2020-04-10T07:12:27.219 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T07:12:27.220 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 28 00 00 00 01 1a 00 00 00 65 72 72 |....(........err| 2020-04-10T07:12:27.220 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 6f 72 20 62 6f 6f 74 73 74 72 61 70 70 69 6e 67 |or bootstrapping| 2020-04-10T07:12:27.220 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 20 72 65 70 6c 61 79 13 1c 90 5e 26 cb 4c 1a 00 | replay...^&.L..| 2020-04-10T07:12:27.220 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000060 2020-04-10T07:12:27.220 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.221 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:status_global_a8183526-4331-4f0d-af4a-7c93c501e927 2020-04-10T07:12:27.221 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (77 bytes) : 2020-04-10T07:12:27.221 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 3f 11 00 00 00 00 00 00 01 |..,....?........| 2020-04-10T07:12:27.221 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 95 c4 de e4 10 00 |................| 2020-04-10T07:12:27.221 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T07:12:27.221 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 15 00 00 00 06 07 00 00 00 73 74 6f |.............sto| 2020-04-10T07:12:27.222 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 70 70 65 64 12 1c 90 5e 06 82 04 18 00 |pped...^.....| 2020-04-10T07:12:27.222 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000004d 2020-04-10T07:12:27.222 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.222 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:status_global_a9af7d9e-1404-4730-9e18-c0418a915346 2020-04-10T07:12:27.222 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (77 bytes) : 2020-04-10T07:12:27.222 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 4b 11 00 00 00 00 00 00 01 |..,....K........| 2020-04-10T07:12:27.222 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 c1 9f 21 4e 10 00 |............!N..| 2020-04-10T07:12:27.223 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T07:12:27.223 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 15 00 00 00 06 07 00 00 00 73 74 6f |.............sto| 2020-04-10T07:12:27.223 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 70 70 65 64 14 1c 90 5e aa 65 77 26 00 |pped...^.ew&.| 2020-04-10T07:12:27.223 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000004d 2020-04-10T07:12:27.223 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.223 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:status_global_b192e243-6abc-405a-b798-b60b9d74691e 2020-04-10T07:12:27.223 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (92 bytes) : 2020-04-10T07:12:27.224 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 51 11 00 00 00 00 00 00 01 |..,....Q........| 2020-04-10T07:12:27.224 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 60 5f b6 16 10 00 |..........`_....| 2020-04-10T07:12:27.224 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T07:12:27.224 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 24 00 00 00 06 16 00 00 00 6c 6f 63 |....$........loc| 2020-04-10T07:12:27.224 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 61 6c 20 69 6d 61 67 65 20 69 73 20 70 72 69 6d |al image is prim| 2020-04-10T07:12:27.224 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 61 72 79 14 1c 90 5e 79 8e fe 21 00 |ary...^y..!.| 2020-04-10T07:12:27.224 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000005c 2020-04-10T07:12:27.224 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.225 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:status_global_b583e294-8b69-4e57-9966-783595c454d6 2020-04-10T07:12:27.225 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (77 bytes) : 2020-04-10T07:12:27.225 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 43 11 00 00 00 00 00 00 01 |..,....C........| 2020-04-10T07:12:27.225 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 09 d2 60 2c 10 00 |............`,..| 2020-04-10T07:12:27.225 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T07:12:27.226 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 15 00 00 00 06 07 00 00 00 73 74 6f |.............sto| 2020-04-10T07:12:27.226 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 70 70 65 64 13 1c 90 5e 52 c6 7b 1f 00 |pped...^R.{..| 2020-04-10T07:12:27.226 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000004d 2020-04-10T07:12:27.226 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.227 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:27.228 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1 mirror ns1 images 2020-04-10T07:12:27.228 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror --namespace ns1 ls -l 2020-04-10T07:12:27.228 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:27.228 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1 mirrorns1 mirror pool info 2020-04-10T07:12:27.228 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror --namespace ns1 mirror pool info 2020-04-10T07:12:27.228 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.228 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster1 mirror ns1 images 2020-04-10T07:12:27.229 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.229 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster1 mirrorns1 mirror pool info 2020-04-10T07:12:27.244 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Mode: image 2020-04-10T07:12:27.249 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:27.250 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1 mirrorns1 mirror pool status 2020-04-10T07:12:27.251 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster1 -p mirror --namespace ns1 mirror pool status --verbose 2020-04-10T07:12:27.251 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.252 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster1 mirrorns1 mirror pool status 2020-04-10T07:12:27.290 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:health: UNKNOWN 2020-04-10T07:12:27.290 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:daemon health: UNKNOWN 2020-04-10T07:12:27.290 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image health: OK 2020-04-10T07:12:27.291 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:images: 0 total 2020-04-10T07:12:27.291 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.291 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:DAEMONS 2020-04-10T07:12:27.291 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: none 2020-04-10T07:12:27.291 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.302 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:27.302 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror --namespace ns1 ls 2020-04-10T07:12:27.302 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:IMAGES 2020-04-10T07:12:27.549 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1 mirror ns1 rbd_mirroring omap vals 2020-04-10T07:12:27.549 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rados --cluster cluster1 -p mirror --namespace ns1 listomapvals rbd_mirroring 2020-04-10T07:12:27.550 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster1 mirror ns1 rbd_mirroring omap vals 2020-04-10T07:12:27.588 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:mirror_mode 2020-04-10T07:12:27.588 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (4 bytes) : 2020-04-10T07:12:27.588 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 00 00 00 |....| 2020-04-10T07:12:27.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000004 2020-04-10T07:12:27.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:mirror_uuid 2020-04-10T07:12:27.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (36 bytes) : 2020-04-10T07:12:27.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 36 66 32 30 33 39 32 30 2d 36 30 39 39 2d 34 32 |6f203920-6099-42| 2020-04-10T07:12:27.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 33 63 2d 61 37 64 64 2d 65 61 35 35 31 32 39 36 |3c-a7dd-ea551296| 2020-04-10T07:12:27.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 64 39 31 61 |d91a| 2020-04-10T07:12:27.590 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000024 2020-04-10T07:12:27.590 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.590 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:27.591 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1 mirror ns2 images 2020-04-10T07:12:27.591 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror --namespace ns2 ls -l 2020-04-10T07:12:27.591 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.591 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster1 mirror ns2 images 2020-04-10T07:12:27.634 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:27.634 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1 mirrorns2 mirror pool info 2020-04-10T07:12:27.634 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror --namespace ns2 mirror pool info 2020-04-10T07:12:27.635 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.635 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster1 mirrorns2 mirror pool info 2020-04-10T07:12:27.679 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Mode: image 2020-04-10T07:12:27.681 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:27.682 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1 mirrorns2 mirror pool status 2020-04-10T07:12:27.682 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster1 -p mirror --namespace ns2 mirror pool status --verbose 2020-04-10T07:12:27.682 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.682 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster1 mirrorns2 mirror pool status 2020-04-10T07:12:27.727 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:health: UNKNOWN 2020-04-10T07:12:27.727 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:daemon health: UNKNOWN 2020-04-10T07:12:27.727 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image health: OK 2020-04-10T07:12:27.728 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:images: 0 total 2020-04-10T07:12:27.728 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.728 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:DAEMONS 2020-04-10T07:12:27.728 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: none 2020-04-10T07:12:27.728 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.738 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:27.738 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:IMAGES 2020-04-10T07:12:27.739 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror --namespace ns2 ls 2020-04-10T07:12:27.780 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1 mirror ns2 rbd_mirroring omap vals 2020-04-10T07:12:27.780 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rados --cluster cluster1 -p mirror --namespace ns2 listomapvals rbd_mirroring 2020-04-10T07:12:27.781 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster1 mirror ns2 rbd_mirroring omap vals 2020-04-10T07:12:27.820 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:mirror_mode 2020-04-10T07:12:27.821 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (4 bytes) : 2020-04-10T07:12:27.821 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 00 00 00 |....| 2020-04-10T07:12:27.821 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000004 2020-04-10T07:12:27.821 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.821 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:mirror_uuid 2020-04-10T07:12:27.821 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (36 bytes) : 2020-04-10T07:12:27.822 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 66 32 36 36 31 30 61 64 2d 66 65 38 32 2d 34 66 |f26610ad-fe82-4f| 2020-04-10T07:12:27.822 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 35 30 2d 38 34 37 30 2d 32 61 35 35 31 31 31 61 |50-8470-2a55111a| 2020-04-10T07:12:27.822 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 38 64 61 63 |8dac| 2020-04-10T07:12:27.822 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000024 2020-04-10T07:12:27.822 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.823 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:27.823 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1 mirror_parent images 2020-04-10T07:12:27.823 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror_parent --namespace ls -l 2020-04-10T07:12:27.824 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.824 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster1 mirror_parent images 2020-04-10T07:12:27.891 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:NAME SIZE PARENT FMT PROT LOCK 2020-04-10T07:12:27.891 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test_parent 128 MiB 2 2020-04-10T07:12:27.891 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test_parent@snap 128 MiB 2 yes 2020-04-10T07:12:27.895 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:27.895 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1 mirror_parent mirror pool info 2020-04-10T07:12:27.895 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror_parent --namespace mirror pool info 2020-04-10T07:12:27.895 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.895 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster1 mirror_parent mirror pool info 2020-04-10T07:12:27.939 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Mode: image 2020-04-10T07:12:27.939 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Site Name: cluster1-DNE 2020-04-10T07:12:27.939 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.939 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Peer Sites: 2020-04-10T07:12:27.939 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.940 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:UUID: 467a3104-15d9-4d31-90d5-8005aa7c9eeb 2020-04-10T07:12:27.940 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Name: cluster2-DNE 2020-04-10T07:12:27.940 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Mirror UUID: e151661d-9adc-4448-8574-890d2847c790 2020-04-10T07:12:27.940 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Direction: rx-tx 2020-04-10T07:12:27.940 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Client: client.mirror 2020-04-10T07:12:27.941 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.942 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster1 mirror_parent mirror pool status 2020-04-10T07:12:27.942 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:27.942 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1 mirror_parent mirror pool status 2020-04-10T07:12:27.942 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster1 -p mirror_parent --namespace mirror pool status --verbose 2020-04-10T07:12:27.987 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:health: UNKNOWN 2020-04-10T07:12:27.987 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:daemon health: UNKNOWN 2020-04-10T07:12:27.987 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image health: OK 2020-04-10T07:12:27.987 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:images: 1 total 2020-04-10T07:12:27.988 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 1 stopped 2020-04-10T07:12:27.988 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:27.988 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:DAEMONS 2020-04-10T07:12:27.988 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: none 2020-04-10T07:12:27.988 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:28.031 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:IMAGES 2020-04-10T07:12:28.031 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test_parent: 2020-04-10T07:12:28.031 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: global_id: dd523e97-79dd-49a7-b2c9-df255e9660d4 2020-04-10T07:12:28.031 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: down+stopped 2020-04-10T07:12:28.032 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: stopped 2020-04-10T07:12:28.032 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 07:11:14 2020-04-10T07:12:28.032 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: peer_sites: 2020-04-10T07:12:28.032 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: name: cluster2-DNE 2020-04-10T07:12:28.032 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+stopped 2020-04-10T07:12:28.032 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: local image is primary 2020-04-10T07:12:28.032 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 07:12:26 2020-04-10T07:12:28.036 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:28.036 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:28.036 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror_parent --namespace ls 2020-04-10T07:12:28.079 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test_parent info 2020-04-10T07:12:28.079 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test_parent info 2020-04-10T07:12:28.080 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror_parent --namespace info test_parent 2020-04-10T07:12:28.148 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:rbd image 'test_parent': 2020-04-10T07:12:28.148 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: size 128 MiB in 32 objects 2020-04-10T07:12:28.149 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: order 22 (4 MiB objects) 2020-04-10T07:12:28.149 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: snapshot_count: 7 2020-04-10T07:12:28.149 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: id: 1154c59c0497 2020-04-10T07:12:28.149 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: block_name_prefix: rbd_data.1154c59c0497 2020-04-10T07:12:28.149 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: format: 2 2020-04-10T07:12:28.150 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: features: layering, exclusive-lock, journaling, non-primary 2020-04-10T07:12:28.150 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: op_features: 2020-04-10T07:12:28.150 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: flags: 2020-04-10T07:12:28.150 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: create_timestamp: Fri Apr 10 07:09:45 2020 2020-04-10T07:12:28.150 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: access_timestamp: Fri Apr 10 07:09:45 2020 2020-04-10T07:12:28.150 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: modify_timestamp: Fri Apr 10 07:09:45 2020 2020-04-10T07:12:28.150 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: journal: 1154c59c0497 2020-04-10T07:12:28.151 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring state: enabled 2020-04-10T07:12:28.151 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring mode: snapshot 2020-04-10T07:12:28.151 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring global id: dd523e97-79dd-49a7-b2c9-df255e9660d4 2020-04-10T07:12:28.151 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring primary: false 2020-04-10T07:12:28.154 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:28.154 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test_parent journal status 2020-04-10T07:12:28.154 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror_parent --namespace journal status --image test_parent 2020-04-10T07:12:28.154 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:28.155 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test_parent journal status 2020-04-10T07:12:28.227 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:minimum_set: 0 2020-04-10T07:12:28.228 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:active_set: 0 2020-04-10T07:12:28.228 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:registered clients: 2020-04-10T07:12:28.228 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: [id=, commit_position=[positions=[]], state=connected] 2020-04-10T07:12:28.233 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:28.233 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster1 mirror_parent rbd_mirroring omap vals 2020-04-10T07:12:28.233 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:28.234 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1 mirror_parent rbd_mirroring omap vals 2020-04-10T07:12:28.234 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rados --cluster cluster1 -p mirror_parent --namespace listomapvals rbd_mirroring 2020-04-10T07:12:28.274 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:global_dd523e97-79dd-49a7-b2c9-df255e9660d4 2020-04-10T07:12:28.274 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (16 bytes) : 2020-04-10T07:12:28.274 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 0c 00 00 00 31 31 35 34 63 35 39 63 30 34 39 37 |....1154c59c0497| 2020-04-10T07:12:28.275 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 2020-04-10T07:12:28.275 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:28.275 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image_1154c59c0497 2020-04-10T07:12:28.275 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (48 bytes) : 2020-04-10T07:12:28.275 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 02 01 2a 00 00 00 24 00 00 00 64 64 35 32 33 65 |..*...$...dd523e| 2020-04-10T07:12:28.275 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 39 37 2d 37 39 64 64 2d 34 39 61 37 2d 62 32 63 |97-79dd-49a7-b2c| 2020-04-10T07:12:28.275 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 39 2d 64 66 32 35 35 65 39 36 36 30 64 34 01 01 |9-df255e9660d4..| 2020-04-10T07:12:28.276 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 2020-04-10T07:12:28.276 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:28.276 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:mirror_mode 2020-04-10T07:12:28.276 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (4 bytes) : 2020-04-10T07:12:28.276 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 00 00 00 |....| 2020-04-10T07:12:28.276 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000004 2020-04-10T07:12:28.276 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:28.277 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:mirror_peer_467a3104-15d9-4d31-90d5-8005aa7c9eeb 2020-04-10T07:12:28.277 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (136 bytes) : 2020-04-10T07:12:28.277 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 02 01 82 00 00 00 24 00 00 00 34 36 37 61 33 31 |......$...467a31| 2020-04-10T07:12:28.277 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 30 34 2d 31 35 64 39 2d 34 64 33 31 2d 39 30 64 |04-15d9-4d31-90d| 2020-04-10T07:12:28.277 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 35 2d 38 30 30 35 61 61 37 63 39 65 65 62 0c 00 |5-8005aa7c9eeb..| 2020-04-10T07:12:28.277 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 63 6c 75 73 74 65 72 32 2d 44 4e 45 0d 00 |..cluster2-DNE..| 2020-04-10T07:12:28.277 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 00 00 63 6c 69 65 6e 74 2e 6d 69 72 72 6f 72 ff |..client.mirror.| 2020-04-10T07:12:28.277 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 ff ff ff ff ff ff ff 02 24 00 00 00 65 31 35 31 |........$...e151| 2020-04-10T07:12:28.278 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000060 36 36 31 64 2d 39 61 64 63 2d 34 34 34 38 2d 38 |661d-9adc-4448-8| 2020-04-10T07:12:28.278 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000070 35 37 34 2d 38 39 30 64 32 38 34 37 63 37 39 30 |574-890d2847c790| 2020-04-10T07:12:28.278 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000080 46 1c 90 5e 6d b1 00 13 |F..^m...| 2020-04-10T07:12:28.278 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000088 2020-04-10T07:12:28.278 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:28.278 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:mirror_uuid 2020-04-10T07:12:28.278 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (36 bytes) : 2020-04-10T07:12:28.279 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 66 61 35 36 37 64 39 33 2d 66 34 36 35 2d 34 37 |fa567d93-f465-47| 2020-04-10T07:12:28.279 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 36 34 2d 39 33 63 37 2d 31 32 39 64 39 39 33 63 |64-93c7-129d993c| 2020-04-10T07:12:28.279 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 66 38 61 31 |f8a1| 2020-04-10T07:12:28.279 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000024 2020-04-10T07:12:28.279 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:28.279 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:remote_status_global_dd523e97-79dd-49a7-b2c9-df255e9660d4_e151661d-9adc-4448-8574-890d2847c790 2020-04-10T07:12:28.279 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (92 bytes) : 2020-04-10T07:12:28.280 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 54 12 00 00 00 00 00 00 01 |..,....T........| 2020-04-10T07:12:28.280 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 60 22 62 98 10 00 |..........`"b...| 2020-04-10T07:12:28.280 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T07:12:28.280 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 24 00 00 00 06 16 00 00 00 6c 6f 63 |....$........loc| 2020-04-10T07:12:28.280 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 61 6c 20 69 6d 61 67 65 20 69 73 20 70 72 69 6d |al image is prim| 2020-04-10T07:12:28.280 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 61 72 79 5a 1c 90 5e b5 3d 4f 12 00 |aryZ..^.=O..| 2020-04-10T07:12:28.280 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000005c 2020-04-10T07:12:28.281 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:28.281 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:status_global_dd523e97-79dd-49a7-b2c9-df255e9660d4 2020-04-10T07:12:28.281 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (77 bytes) : 2020-04-10T07:12:28.281 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 54 11 00 00 00 00 00 00 01 |..,....T........| 2020-04-10T07:12:28.281 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 82 7b f2 0b 10 00 |...........{....| 2020-04-10T07:12:28.281 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T07:12:28.281 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 15 00 00 00 06 07 00 00 00 73 74 6f |.............sto| 2020-04-10T07:12:28.282 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 70 70 65 64 12 1c 90 5e 3d 19 b6 14 00 |pped...^=....| 2020-04-10T07:12:28.282 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000004d 2020-04-10T07:12:28.282 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:28.282 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:28.283 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster1 mirror_parent ns1 images 2020-04-10T07:12:28.283 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:28.283 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1 mirror_parent ns1 images 2020-04-10T07:12:28.283 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror_parent --namespace ns1 ls -l 2020-04-10T07:12:28.314 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: namespace 'ns1' does not exist. 2020-04-10T07:12:28.314 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: listing images failed: (2) No such file or directory 2020-04-10T07:12:28.317 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:28.318 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster1 mirror_parentns1 mirror pool info 2020-04-10T07:12:28.318 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:28.318 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1 mirror_parentns1 mirror pool info 2020-04-10T07:12:28.318 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror_parent --namespace ns1 mirror pool info 2020-04-10T07:12:28.354 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: namespace 'ns1' does not exist. 2020-04-10T07:12:28.356 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:28.356 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1 mirror_parentns1 mirror pool status 2020-04-10T07:12:28.357 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster1 -p mirror_parent --namespace ns1 mirror pool status --verbose 2020-04-10T07:12:28.357 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:28.357 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster1 mirror_parentns1 mirror pool status 2020-04-10T07:12:28.395 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: namespace 'ns1' does not exist. 2020-04-10T07:12:28.398 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:28.398 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror_parent --namespace ns1 ls 2020-04-10T07:12:28.398 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:28.439 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster1 mirror_parent ns1 rbd_mirroring omap vals 2020-04-10T07:12:28.439 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1 mirror_parent ns1 rbd_mirroring omap vals 2020-04-10T07:12:28.439 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rados --cluster cluster1 -p mirror_parent --namespace ns1 listomapvals rbd_mirroring 2020-04-10T07:12:28.478 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:error getting omap keys mirror_parent/rbd_mirroring: (2) No such file or directory 2020-04-10T07:12:28.480 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:28.480 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1 mirror_parent ns2 images 2020-04-10T07:12:28.481 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror_parent --namespace ns2 ls -l 2020-04-10T07:12:28.481 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:28.481 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster1 mirror_parent ns2 images 2020-04-10T07:12:28.522 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: namespace 'ns2' does not exist. 2020-04-10T07:12:28.522 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: listing images failed: (2) No such file or directory 2020-04-10T07:12:28.522 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:28.522 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1 mirror_parentns2 mirror pool info 2020-04-10T07:12:28.522 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror_parent --namespace ns2 mirror pool info 2020-04-10T07:12:28.523 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:28.523 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster1 mirror_parentns2 mirror pool info 2020-04-10T07:12:28.561 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: namespace 'ns2' does not exist. 2020-04-10T07:12:28.565 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:28.565 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1 mirror_parentns2 mirror pool status 2020-04-10T07:12:28.565 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster1 -p mirror_parent --namespace ns2 mirror pool status --verbose 2020-04-10T07:12:28.565 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:28.566 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster1 mirror_parentns2 mirror pool status 2020-04-10T07:12:28.602 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: namespace 'ns2' does not exist. 2020-04-10T07:12:28.604 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:28.604 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:28.605 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror_parent --namespace ns2 ls 2020-04-10T07:12:28.644 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster1 mirror_parent ns2 rbd_mirroring omap vals 2020-04-10T07:12:28.645 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1 mirror_parent ns2 rbd_mirroring omap vals 2020-04-10T07:12:28.645 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rados --cluster cluster1 -p mirror_parent --namespace ns2 listomapvals rbd_mirroring 2020-04-10T07:12:28.687 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:error getting omap keys mirror_parent/rbd_mirroring: (2) No such file or directory 2020-04-10T07:12:28.689 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:28.689 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster2 status 2020-04-10T07:12:28.689 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:28.689 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2 status 2020-04-10T07:12:28.690 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= ceph --cluster cluster2 -s 2020-04-10T07:12:29.107 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: cluster: 2020-04-10T07:12:29.107 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: id: b4476b22-7943-481f-a1ec-34bc78d4adbc 2020-04-10T07:12:29.107 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: health: HEALTH_OK 2020-04-10T07:12:29.107 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:29.107 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: services: 2020-04-10T07:12:29.108 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mon: 1 daemons, quorum a (age 7m) 2020-04-10T07:12:29.108 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mgr: x(active, since 7m) 2020-04-10T07:12:29.108 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: osd: 3 osds: 3 up (since 7m), 3 in (since 7m) 2020-04-10T07:12:29.108 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: rbd-mirror: 2 daemons active (4491, 4499) 2020-04-10T07:12:29.108 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:29.108 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: task status: 2020-04-10T07:12:29.109 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:29.109 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: data: 2020-04-10T07:12:29.109 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: pools: 4 pools, 137 pgs 2020-04-10T07:12:29.109 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: objects: 574 objects, 825 MiB 2020-04-10T07:12:29.109 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: usage: 3.0 GiB used, 267 GiB / 270 GiB avail 2020-04-10T07:12:29.109 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: pgs: 137 active+clean 2020-04-10T07:12:29.109 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:29.110 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: io: 2020-04-10T07:12:29.110 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: client: 19 KiB/s rd, 2.2 KiB/s wr, 19 op/s rd, 2 op/s wr 2020-04-10T07:12:29.110 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:29.125 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= ceph --cluster cluster2 service dump 2020-04-10T07:12:29.444 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:{ 2020-04-10T07:12:29.444 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "epoch": 4, 2020-04-10T07:12:29.444 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "modified": "2020-04-10T07:11:21.915914+0000", 2020-04-10T07:12:29.444 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "services": { 2020-04-10T07:12:29.444 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "rbd-mirror": { 2020-04-10T07:12:29.445 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "daemons": { 2020-04-10T07:12:29.445 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "summary": "", 2020-04-10T07:12:29.445 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "4491": { 2020-04-10T07:12:29.445 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "start_epoch": 2, 2020-04-10T07:12:29.445 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "start_stamp": "2020-04-10T07:08:06.046076+0000", 2020-04-10T07:12:29.445 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "gid": 4491, 2020-04-10T07:12:29.445 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "addr": "172.21.15.204:0/4237060419", 2020-04-10T07:12:29.445 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "metadata": { 2020-04-10T07:12:29.446 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "arch": "x86_64", 2020-04-10T07:12:29.446 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "ceph_release": "octopus", 2020-04-10T07:12:29.446 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "ceph_version": "ceph version 15.2.0-148-g39357d2 (39357d2a782a02d1ea09c76cbe7bc6f222a1127a) octopus (stable)", 2020-04-10T07:12:29.446 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "ceph_version_short": "15.2.0-148-g39357d2", 2020-04-10T07:12:29.446 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz", 2020-04-10T07:12:29.446 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "distro": "ubuntu", 2020-04-10T07:12:29.446 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "distro_description": "Ubuntu 18.04.4 LTS", 2020-04-10T07:12:29.447 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "distro_version": "18.04", 2020-04-10T07:12:29.447 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "hostname": "smithi204", 2020-04-10T07:12:29.447 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "id": "mirror.2", 2020-04-10T07:12:29.447 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "instance_id": "4491", 2020-04-10T07:12:29.447 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "kernel_description": "#97-Ubuntu SMP Wed Apr 1 03:25:46 UTC 2020", 2020-04-10T07:12:29.447 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "kernel_version": "4.15.0-96-generic", 2020-04-10T07:12:29.447 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "mem_swap_kb": "2097148", 2020-04-10T07:12:29.448 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "mem_total_kb": "32823944", 2020-04-10T07:12:29.448 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "os": "Linux" 2020-04-10T07:12:29.448 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: }, 2020-04-10T07:12:29.448 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "task_status": {} 2020-04-10T07:12:29.448 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: }, 2020-04-10T07:12:29.448 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "4499": { 2020-04-10T07:12:29.448 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "start_epoch": 2, 2020-04-10T07:12:29.448 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "start_stamp": "2020-04-10T07:08:06.075848+0000", 2020-04-10T07:12:29.449 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "gid": 4499, 2020-04-10T07:12:29.449 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "addr": "172.21.15.204:0/2107014639", 2020-04-10T07:12:29.449 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "metadata": { 2020-04-10T07:12:29.449 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "arch": "x86_64", 2020-04-10T07:12:29.449 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "ceph_release": "octopus", 2020-04-10T07:12:29.449 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "ceph_version": "ceph version 15.2.0-148-g39357d2 (39357d2a782a02d1ea09c76cbe7bc6f222a1127a) octopus (stable)", 2020-04-10T07:12:29.449 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "ceph_version_short": "15.2.0-148-g39357d2", 2020-04-10T07:12:29.450 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz", 2020-04-10T07:12:29.450 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "distro": "ubuntu", 2020-04-10T07:12:29.450 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "distro_description": "Ubuntu 18.04.4 LTS", 2020-04-10T07:12:29.450 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "distro_version": "18.04", 2020-04-10T07:12:29.450 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "hostname": "smithi204", 2020-04-10T07:12:29.450 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "id": "mirror.3", 2020-04-10T07:12:29.450 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "instance_id": "4499", 2020-04-10T07:12:29.451 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "kernel_description": "#97-Ubuntu SMP Wed Apr 1 03:25:46 UTC 2020", 2020-04-10T07:12:29.451 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "kernel_version": "4.15.0-96-generic", 2020-04-10T07:12:29.451 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "mem_swap_kb": "2097148", 2020-04-10T07:12:29.451 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "mem_total_kb": "32823944", 2020-04-10T07:12:29.451 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "os": "Linux" 2020-04-10T07:12:29.451 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: }, 2020-04-10T07:12:29.451 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "task_status": {} 2020-04-10T07:12:29.451 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: } 2020-04-10T07:12:29.452 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: } 2020-04-10T07:12:29.452 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: } 2020-04-10T07:12:29.452 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: } 2020-04-10T07:12:29.452 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:} 2020-04-10T07:12:29.456 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= ceph --cluster cluster2 service status 2020-04-10T07:12:29.778 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:{ 2020-04-10T07:12:29.778 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "rbd-mirror": { 2020-04-10T07:12:29.778 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "4491": { 2020-04-10T07:12:29.778 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "status_stamp": "2020-04-10T07:12:26.054589+0000", 2020-04-10T07:12:29.779 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "last_beacon": "2020-04-10T07:12:26.054589+0000", 2020-04-10T07:12:29.779 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "status": { 2020-04-10T07:12:29.779 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "json": "{\"3\":{\"name\":\"mirror\",\"callouts\":{},\"image_assigned_count\":4,\"image_error_count\":0,\"image_warning_count\":0,\"instance_id\":\"4502\",\"namespaces\":{\"ns1\":{\"image_assigned_count\":0,\"image_error_count\":0,\"image_warning_count\":0},\"ns2\":{\"image_assigned_count\":0,\"image_error_count\":0,\"image_warning_count\":0}}},\"4\":{\"name\":\"mirror_parent\",\"callouts\":{}}}" 2020-04-10T07:12:29.779 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: } 2020-04-10T07:12:29.779 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: }, 2020-04-10T07:12:29.779 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "4499": { 2020-04-10T07:12:29.779 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "status_stamp": "2020-04-10T07:12:26.084094+0000", 2020-04-10T07:12:29.780 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "last_beacon": "2020-04-10T07:12:26.084094+0000", 2020-04-10T07:12:29.780 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "status": { 2020-04-10T07:12:29.780 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "json": "{\"3\":{\"name\":\"mirror\",\"callouts\":{},\"image_assigned_count\":5,\"image_error_count\":1,\"image_local_count\":9,\"image_remote_count\":8,\"image_warning_count\":0,\"instance_id\":\"4510\",\"leader\":true,\"namespaces\":{\"ns1\":{\"image_assigned_count\":0,\"image_error_count\":0,\"image_local_count\":0,\"image_remote_count\":0,\"image_warning_count\":0},\"ns2\":{\"image_assigned_count\":0,\"image_error_count\":0,\"image_local_count\":0,\"image_remote_count\":0,\"image_warning_count\":0}}},\"4\":{\"name\":\"mirror_parent\",\"callouts\":{},\"image_assigned_count\":1,\"image_error_count\":0,\"image_local_count\":1,\"image_remote_count\":1,\"image_warning_count\":0,\"instance_id\":\"4522\",\"leader\":true}}" 2020-04-10T07:12:29.780 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: } 2020-04-10T07:12:29.780 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: } 2020-04-10T07:12:29.780 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: } 2020-04-10T07:12:29.781 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:} 2020-04-10T07:12:29.789 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:29.790 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster2 mirror images 2020-04-10T07:12:29.790 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:29.790 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2 mirror images 2020-04-10T07:12:29.790 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror --namespace ls -l 2020-04-10T07:12:29.880 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:NAME SIZE PARENT FMT PROT LOCK 2020-04-10T07:12:29.880 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test 128 MiB 2 2020-04-10T07:12:29.880 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test1 128 MiB 2 2020-04-10T07:12:29.880 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test2 128 MiB 2 2020-04-10T07:12:29.880 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test2@snap1 128 MiB 2 2020-04-10T07:12:29.881 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test2@snap2 128 MiB 2 2020-04-10T07:12:29.881 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test2@snap_rename_20 128 MiB 2 2020-04-10T07:12:29.881 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test4 128 MiB 2 2020-04-10T07:12:29.881 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test4@snap1 128 MiB 2 yes 2020-04-10T07:12:29.881 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test4@snap2 128 MiB 2 yes 2020-04-10T07:12:29.881 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test_clone 128 MiB mirror_parent/test_parent@snap 2 2020-04-10T07:12:29.881 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test_clone1 128 MiB mirror_parent/test_parent@snap 2 excl 2020-04-10T07:12:29.882 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test_clone_v1 128 MiB mirror_parent/test_parent@snap 2 2020-04-10T07:12:29.882 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test_data_pool 128 MiB 2 2020-04-10T07:12:29.882 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test_data_pool@snap1 128 MiB 2 2020-04-10T07:12:29.882 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test_data_pool@snap2 128 MiB 2 2020-04-10T07:12:29.882 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test_force_promote 128 MiB 2 2020-04-10T07:12:29.883 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:29.883 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2 mirror mirror pool info 2020-04-10T07:12:29.883 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror --namespace mirror pool info 2020-04-10T07:12:29.883 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:29.883 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster2 mirror mirror pool info 2020-04-10T07:12:29.914 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Mode: image 2020-04-10T07:12:29.914 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Site Name: cluster2-DNE 2020-04-10T07:12:29.915 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:29.915 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Peer Sites: 2020-04-10T07:12:29.915 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:29.915 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:UUID: 90b6cfa2-5481-4edf-b462-5494aab62f6f 2020-04-10T07:12:29.915 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Name: cluster1-DNE 2020-04-10T07:12:29.915 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Mirror UUID: 620ddb00-0f49-4758-a216-8d9cc890d0fa 2020-04-10T07:12:29.916 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Direction: rx-tx 2020-04-10T07:12:29.916 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Client: client.mirror 2020-04-10T07:12:29.917 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:29.917 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2 mirror mirror pool status 2020-04-10T07:12:29.917 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster2 -p mirror --namespace mirror pool status --verbose 2020-04-10T07:12:29.918 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:29.918 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster2 mirror mirror pool status 2020-04-10T07:12:29.958 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:health: ERROR 2020-04-10T07:12:29.958 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:daemon health: OK 2020-04-10T07:12:29.958 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image health: ERROR 2020-04-10T07:12:29.958 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:images: 9 total 2020-04-10T07:12:29.958 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 8 unknown 2020-04-10T07:12:29.959 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 1 error 2020-04-10T07:12:29.959 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:29.959 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:DAEMONS 2020-04-10T07:12:29.959 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:service 4491: 2020-04-10T07:12:29.959 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: instance_id: 4502 2020-04-10T07:12:29.959 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: client_id: mirror.2 2020-04-10T07:12:29.959 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: hostname: smithi204 2020-04-10T07:12:29.960 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: version: 15.2.0-148-g39357d2 2020-04-10T07:12:29.960 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: leader: false 2020-04-10T07:12:29.960 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: health: OK 2020-04-10T07:12:29.960 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:29.960 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:service 4499: 2020-04-10T07:12:29.960 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: instance_id: 4510 2020-04-10T07:12:29.961 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: client_id: mirror.3 2020-04-10T07:12:29.961 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: hostname: smithi204 2020-04-10T07:12:29.961 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: version: 15.2.0-148-g39357d2 2020-04-10T07:12:29.961 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: leader: true 2020-04-10T07:12:29.961 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: health: OK 2020-04-10T07:12:29.961 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:29.961 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:30.005 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:IMAGES 2020-04-10T07:12:30.006 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test: 2020-04-10T07:12:30.006 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: global_id: a9af7d9e-1404-4730-9e18-c0418a915346 2020-04-10T07:12:30.006 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+stopped 2020-04-10T07:12:30.006 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: local image is primary 2020-04-10T07:12:30.006 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: service: mirror.3 on smithi204 2020-04-10T07:12:30.006 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 07:12:26 2020-04-10T07:12:30.007 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: peer_sites: 2020-04-10T07:12:30.007 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: name: cluster1-DNE 2020-04-10T07:12:30.007 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: down+stopped 2020-04-10T07:12:30.007 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: stopped 2020-04-10T07:12:30.007 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 07:11:16 2020-04-10T07:12:30.020 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:30.020 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test1: 2020-04-10T07:12:30.020 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: global_id: 56419286-13e9-49f6-a20d-2abe6ede083e 2020-04-10T07:12:30.020 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+stopped 2020-04-10T07:12:30.020 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: local image is primary 2020-04-10T07:12:30.020 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: service: mirror.3 on smithi204 2020-04-10T07:12:30.021 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 07:12:26 2020-04-10T07:12:30.021 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: peer_sites: 2020-04-10T07:12:30.021 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: name: cluster1-DNE 2020-04-10T07:12:30.021 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: down+stopped 2020-04-10T07:12:30.021 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: stopped 2020-04-10T07:12:30.021 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 07:11:14 2020-04-10T07:12:30.021 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:30.022 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test2: 2020-04-10T07:12:30.022 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: global_id: b583e294-8b69-4e57-9966-783595c454d6 2020-04-10T07:12:30.022 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+stopped 2020-04-10T07:12:30.022 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: local image is primary 2020-04-10T07:12:30.022 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: service: mirror.2 on smithi204 2020-04-10T07:12:30.022 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 07:12:26 2020-04-10T07:12:30.023 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: peer_sites: 2020-04-10T07:12:30.023 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: name: cluster1-DNE 2020-04-10T07:12:30.023 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: down+stopped 2020-04-10T07:12:30.023 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: stopped 2020-04-10T07:12:30.023 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 07:11:15 2020-04-10T07:12:30.029 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:30.029 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test_clone: 2020-04-10T07:12:30.029 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: global_id: 003078e2-ed6c-4f36-8bc5-ce0c44ef25d5 2020-04-10T07:12:30.029 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+stopped 2020-04-10T07:12:30.029 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: local image is primary 2020-04-10T07:12:30.030 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: service: mirror.3 on smithi204 2020-04-10T07:12:30.030 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 07:12:26 2020-04-10T07:12:30.030 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: peer_sites: 2020-04-10T07:12:30.030 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: name: cluster1-DNE 2020-04-10T07:12:30.030 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: down+stopped 2020-04-10T07:12:30.030 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: stopped 2020-04-10T07:12:30.030 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 07:11:14 2020-04-10T07:12:30.036 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:30.036 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test_clone1: 2020-04-10T07:12:30.036 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: global_id: 4b1961cb-5406-443f-90f1-3627ac4ffffd 2020-04-10T07:12:30.036 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+replaying 2020-04-10T07:12:30.036 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: replaying, {"replay_state": "idle", "remote_snapshot_timestamp": 1586502598, "local_snapshot_timestamp": 1586502598} 2020-04-10T07:12:30.037 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: service: mirror.2 on smithi204 2020-04-10T07:12:30.037 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 07:12:06 2020-04-10T07:12:30.037 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: peer_sites: 2020-04-10T07:12:30.037 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: name: cluster1-DNE 2020-04-10T07:12:30.037 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: down+stopped 2020-04-10T07:12:30.037 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: local image is primary 2020-04-10T07:12:30.037 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 07:11:15 2020-04-10T07:12:30.038 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:30.038 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test_clone_v1: 2020-04-10T07:12:30.038 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: global_id: 70d86755-f73a-4bbe-9493-1b79a191cc0c 2020-04-10T07:12:30.038 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+stopped 2020-04-10T07:12:30.038 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: local image is primary 2020-04-10T07:12:30.038 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: service: mirror.2 on smithi204 2020-04-10T07:12:30.038 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 07:12:26 2020-04-10T07:12:30.039 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: peer_sites: 2020-04-10T07:12:30.039 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: name: cluster1-DNE 2020-04-10T07:12:30.039 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: down+stopped 2020-04-10T07:12:30.039 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: stopped 2020-04-10T07:12:30.039 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 07:11:16 2020-04-10T07:12:30.039 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:30.039 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test_data_pool: 2020-04-10T07:12:30.040 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: global_id: a8183526-4331-4f0d-af4a-7c93c501e927 2020-04-10T07:12:30.040 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+stopped 2020-04-10T07:12:30.040 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: local image is primary 2020-04-10T07:12:30.040 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: service: mirror.3 on smithi204 2020-04-10T07:12:30.040 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 07:12:26 2020-04-10T07:12:30.040 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: peer_sites: 2020-04-10T07:12:30.040 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: name: cluster1-DNE 2020-04-10T07:12:30.041 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: down+stopped 2020-04-10T07:12:30.041 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: stopped 2020-04-10T07:12:30.041 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 07:11:14 2020-04-10T07:12:30.041 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:30.041 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test_force_promote: 2020-04-10T07:12:30.041 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: global_id: b192e243-6abc-405a-b798-b60b9d74691e 2020-04-10T07:12:30.041 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+stopped 2020-04-10T07:12:30.042 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: local image is primary 2020-04-10T07:12:30.042 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: service: mirror.2 on smithi204 2020-04-10T07:12:30.042 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 07:12:26 2020-04-10T07:12:30.042 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: peer_sites: 2020-04-10T07:12:30.042 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: name: cluster1-DNE 2020-04-10T07:12:30.042 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: down+stopped 2020-04-10T07:12:30.042 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: local image is primary 2020-04-10T07:12:30.042 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 07:11:16 2020-04-10T07:12:30.043 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:30.043 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror --namespace ls 2020-04-10T07:12:30.043 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:30.070 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test info 2020-04-10T07:12:30.070 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror --namespace info test 2020-04-10T07:12:30.071 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test info 2020-04-10T07:12:30.130 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:rbd image 'test': 2020-04-10T07:12:30.130 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: size 128 MiB in 32 objects 2020-04-10T07:12:30.130 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: order 22 (4 MiB objects) 2020-04-10T07:12:30.130 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: snapshot_count: 11 2020-04-10T07:12:30.131 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: id: 10e54538c85b 2020-04-10T07:12:30.131 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: block_name_prefix: rbd_data.10e54538c85b 2020-04-10T07:12:30.131 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: format: 2 2020-04-10T07:12:30.131 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: features: layering, exclusive-lock, journaling 2020-04-10T07:12:30.131 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: op_features: 2020-04-10T07:12:30.131 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: flags: 2020-04-10T07:12:30.131 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: create_timestamp: Fri Apr 10 07:06:26 2020 2020-04-10T07:12:30.132 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: access_timestamp: Fri Apr 10 07:08:21 2020 2020-04-10T07:12:30.132 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: modify_timestamp: Fri Apr 10 07:08:16 2020 2020-04-10T07:12:30.132 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: journal: 10e54538c85b 2020-04-10T07:12:30.132 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring state: enabled 2020-04-10T07:12:30.132 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring mode: snapshot 2020-04-10T07:12:30.132 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring global id: a9af7d9e-1404-4730-9e18-c0418a915346 2020-04-10T07:12:30.132 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring primary: true 2020-04-10T07:12:30.134 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:30.134 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test journal status 2020-04-10T07:12:30.134 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror --namespace journal status --image test 2020-04-10T07:12:30.135 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:30.135 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test journal status 2020-04-10T07:12:30.203 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:minimum_set: 0 2020-04-10T07:12:30.203 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:active_set: 0 2020-04-10T07:12:30.204 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:registered clients: 2020-04-10T07:12:30.204 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: [id=, commit_position=[positions=[[object_number=1, tag_tid=32, entry_tid=1], [object_number=0, tag_tid=32, entry_tid=0], [object_number=3, tag_tid=30, entry_tid=99], [object_number=2, tag_tid=30, entry_tid=98]]], state=connected] 2020-04-10T07:12:30.206 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:30.207 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test1 info 2020-04-10T07:12:30.207 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror --namespace info test1 2020-04-10T07:12:30.207 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:30.207 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test1 info 2020-04-10T07:12:30.260 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:rbd image 'test1': 2020-04-10T07:12:30.260 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: size 128 MiB in 32 objects 2020-04-10T07:12:30.261 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: order 22 (4 MiB objects) 2020-04-10T07:12:30.261 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: snapshot_count: 1 2020-04-10T07:12:30.261 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: id: 111968743338 2020-04-10T07:12:30.261 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: block_name_prefix: rbd_data.111968743338 2020-04-10T07:12:30.261 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: format: 2 2020-04-10T07:12:30.262 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: features: layering, exclusive-lock, journaling 2020-04-10T07:12:30.262 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: op_features: 2020-04-10T07:12:30.262 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: flags: 2020-04-10T07:12:30.262 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: create_timestamp: Fri Apr 10 07:06:38 2020 2020-04-10T07:12:30.262 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: access_timestamp: Fri Apr 10 07:06:38 2020 2020-04-10T07:12:30.262 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: modify_timestamp: Fri Apr 10 07:06:38 2020 2020-04-10T07:12:30.263 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: journal: 111968743338 2020-04-10T07:12:30.263 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring state: enabled 2020-04-10T07:12:30.263 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring mode: snapshot 2020-04-10T07:12:30.263 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring global id: 56419286-13e9-49f6-a20d-2abe6ede083e 2020-04-10T07:12:30.263 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring primary: true 2020-04-10T07:12:30.265 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:30.265 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test1 journal status 2020-04-10T07:12:30.265 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror --namespace journal status --image test1 2020-04-10T07:12:30.265 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:30.265 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test1 journal status 2020-04-10T07:12:30.332 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:minimum_set: 0 2020-04-10T07:12:30.332 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:active_set: 0 2020-04-10T07:12:30.332 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:registered clients: 2020-04-10T07:12:30.332 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: [id=, commit_position=[positions=[[object_number=1, tag_tid=4, entry_tid=1], [object_number=0, tag_tid=4, entry_tid=0], [object_number=3, tag_tid=2, entry_tid=99], [object_number=2, tag_tid=2, entry_tid=98]]], state=connected] 2020-04-10T07:12:30.335 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:30.336 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test2 info 2020-04-10T07:12:30.336 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:30.336 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test2 info 2020-04-10T07:12:30.337 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror --namespace info test2 2020-04-10T07:12:30.391 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:rbd image 'test2': 2020-04-10T07:12:30.391 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: size 128 MiB in 32 objects 2020-04-10T07:12:30.391 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: order 22 (4 MiB objects) 2020-04-10T07:12:30.391 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: snapshot_count: 4 2020-04-10T07:12:30.392 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: id: 12eff4a76192 2020-04-10T07:12:30.392 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: block_name_prefix: rbd_data.12eff4a76192 2020-04-10T07:12:30.392 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: format: 2 2020-04-10T07:12:30.392 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: features: layering, exclusive-lock, journaling 2020-04-10T07:12:30.392 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: op_features: 2020-04-10T07:12:30.392 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: flags: 2020-04-10T07:12:30.392 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: create_timestamp: Fri Apr 10 07:10:32 2020 2020-04-10T07:12:30.392 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: access_timestamp: Fri Apr 10 07:10:32 2020 2020-04-10T07:12:30.393 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: modify_timestamp: Fri Apr 10 07:10:32 2020 2020-04-10T07:12:30.393 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: journal: 12eff4a76192 2020-04-10T07:12:30.393 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring state: enabled 2020-04-10T07:12:30.393 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring mode: snapshot 2020-04-10T07:12:30.393 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring global id: b583e294-8b69-4e57-9966-783595c454d6 2020-04-10T07:12:30.393 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring primary: true 2020-04-10T07:12:30.396 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:30.396 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test2 journal status 2020-04-10T07:12:30.396 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror --namespace journal status --image test2 2020-04-10T07:12:30.397 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:30.397 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test2 journal status 2020-04-10T07:12:30.466 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:minimum_set: 0 2020-04-10T07:12:30.466 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:active_set: 0 2020-04-10T07:12:30.466 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:registered clients: 2020-04-10T07:12:30.467 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: [id=, commit_position=[positions=[[object_number=1, tag_tid=31, entry_tid=1], [object_number=0, tag_tid=31, entry_tid=0], [object_number=3, tag_tid=7, entry_tid=3], [object_number=2, tag_tid=7, entry_tid=2]]], state=connected] 2020-04-10T07:12:30.470 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:30.470 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test4 info 2020-04-10T07:12:30.470 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror --namespace info test4 2020-04-10T07:12:30.470 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:30.470 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test4 info 2020-04-10T07:12:30.524 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:rbd image 'test4': 2020-04-10T07:12:30.525 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: size 128 MiB in 32 objects 2020-04-10T07:12:30.525 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: order 22 (4 MiB objects) 2020-04-10T07:12:30.525 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: snapshot_count: 2 2020-04-10T07:12:30.525 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: id: 1328aa917c4d 2020-04-10T07:12:30.525 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: block_name_prefix: rbd_data.1328aa917c4d 2020-04-10T07:12:30.525 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: format: 2 2020-04-10T07:12:30.525 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: features: layering, exclusive-lock, journaling 2020-04-10T07:12:30.526 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: op_features: 2020-04-10T07:12:30.526 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: flags: 2020-04-10T07:12:30.526 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: create_timestamp: Fri Apr 10 07:10:44 2020 2020-04-10T07:12:30.526 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: access_timestamp: Fri Apr 10 07:10:44 2020 2020-04-10T07:12:30.526 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: modify_timestamp: Fri Apr 10 07:10:44 2020 2020-04-10T07:12:30.526 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: journal: 1328aa917c4d 2020-04-10T07:12:30.526 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring state: disabled 2020-04-10T07:12:30.530 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:30.530 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test4 journal status 2020-04-10T07:12:30.530 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror --namespace journal status --image test4 2020-04-10T07:12:30.530 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:30.531 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test4 journal status 2020-04-10T07:12:30.599 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:minimum_set: 0 2020-04-10T07:12:30.599 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:active_set: 0 2020-04-10T07:12:30.599 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:registered clients: 2020-04-10T07:12:30.599 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: [id=, commit_position=[positions=[[object_number=3, tag_tid=9, entry_tid=3], [object_number=2, tag_tid=9, entry_tid=2], [object_number=1, tag_tid=9, entry_tid=1], [object_number=0, tag_tid=9, entry_tid=0]]], state=connected] 2020-04-10T07:12:30.602 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:30.602 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test_clone info 2020-04-10T07:12:30.602 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror --namespace info test_clone 2020-04-10T07:12:30.602 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:30.603 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test_clone info 2020-04-10T07:12:30.672 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:rbd image 'test_clone': 2020-04-10T07:12:30.673 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: size 128 MiB in 32 objects 2020-04-10T07:12:30.673 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: order 22 (4 MiB objects) 2020-04-10T07:12:30.673 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: snapshot_count: 1 2020-04-10T07:12:30.673 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: id: 1268616d9c91 2020-04-10T07:12:30.673 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: block_name_prefix: rbd_data.1268616d9c91 2020-04-10T07:12:30.673 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: format: 2 2020-04-10T07:12:30.674 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: features: layering, exclusive-lock, journaling 2020-04-10T07:12:30.674 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: op_features: 2020-04-10T07:12:30.674 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: flags: 2020-04-10T07:12:30.674 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: create_timestamp: Fri Apr 10 07:09:44 2020 2020-04-10T07:12:30.674 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: access_timestamp: Fri Apr 10 07:09:44 2020 2020-04-10T07:12:30.674 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: modify_timestamp: Fri Apr 10 07:09:44 2020 2020-04-10T07:12:30.675 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: parent: mirror_parent/test_parent@snap 2020-04-10T07:12:30.675 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: overlap: 128 MiB 2020-04-10T07:12:30.675 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: journal: 1268616d9c91 2020-04-10T07:12:30.675 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring state: enabled 2020-04-10T07:12:30.675 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring mode: snapshot 2020-04-10T07:12:30.675 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring global id: 003078e2-ed6c-4f36-8bc5-ce0c44ef25d5 2020-04-10T07:12:30.676 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring primary: true 2020-04-10T07:12:30.678 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:30.679 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test_clone journal status 2020-04-10T07:12:30.679 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:30.679 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test_clone journal status 2020-04-10T07:12:30.679 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror --namespace journal status --image test_clone 2020-04-10T07:12:30.757 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:minimum_set: 0 2020-04-10T07:12:30.757 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:active_set: 0 2020-04-10T07:12:30.758 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:registered clients: 2020-04-10T07:12:30.758 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: [id=, commit_position=[positions=[[object_number=1, tag_tid=4, entry_tid=1], [object_number=0, tag_tid=4, entry_tid=0], [object_number=3, tag_tid=1, entry_tid=99], [object_number=2, tag_tid=1, entry_tid=98]]], state=connected] 2020-04-10T07:12:30.761 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:30.761 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test_clone1 info 2020-04-10T07:12:30.761 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror --namespace info test_clone1 2020-04-10T07:12:30.762 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:30.762 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test_clone1 info 2020-04-10T07:12:30.833 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:rbd image 'test_clone1': 2020-04-10T07:12:30.833 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: size 128 MiB in 32 objects 2020-04-10T07:12:30.833 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: order 22 (4 MiB objects) 2020-04-10T07:12:30.833 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: snapshot_count: 1 2020-04-10T07:12:30.833 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: id: 1189d1969f13 2020-04-10T07:12:30.834 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: block_name_prefix: rbd_data.1189d1969f13 2020-04-10T07:12:30.834 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: format: 2 2020-04-10T07:12:30.834 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: features: layering, exclusive-lock, journaling, non-primary 2020-04-10T07:12:30.834 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: op_features: 2020-04-10T07:12:30.834 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: flags: 2020-04-10T07:12:30.834 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: create_timestamp: Fri Apr 10 07:10:00 2020 2020-04-10T07:12:30.834 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: access_timestamp: Fri Apr 10 07:10:00 2020 2020-04-10T07:12:30.835 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: modify_timestamp: Fri Apr 10 07:10:00 2020 2020-04-10T07:12:30.835 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: parent: mirror_parent/test_parent@snap 2020-04-10T07:12:30.835 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: overlap: 128 MiB 2020-04-10T07:12:30.835 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: journal: 1189d1969f13 2020-04-10T07:12:30.835 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring state: enabled 2020-04-10T07:12:30.836 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring mode: snapshot 2020-04-10T07:12:30.836 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring global id: 4b1961cb-5406-443f-90f1-3627ac4ffffd 2020-04-10T07:12:30.836 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring primary: false 2020-04-10T07:12:30.838 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:30.838 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test_clone1 journal status 2020-04-10T07:12:30.838 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:30.838 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test_clone1 journal status 2020-04-10T07:12:30.839 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror --namespace journal status --image test_clone1 2020-04-10T07:12:30.917 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:minimum_set: 0 2020-04-10T07:12:30.919 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:active_set: 0 2020-04-10T07:12:30.919 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:registered clients: 2020-04-10T07:12:30.920 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: [id=, commit_position=[positions=[]], state=connected] 2020-04-10T07:12:30.921 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:30.921 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test_clone_v1 info 2020-04-10T07:12:30.921 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:30.922 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test_clone_v1 info 2020-04-10T07:12:30.922 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror --namespace info test_clone_v1 2020-04-10T07:12:30.996 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:rbd image 'test_clone_v1': 2020-04-10T07:12:30.997 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: size 128 MiB in 32 objects 2020-04-10T07:12:30.997 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: order 22 (4 MiB objects) 2020-04-10T07:12:30.997 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: snapshot_count: 1 2020-04-10T07:12:30.997 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: id: 1287c927b049 2020-04-10T07:12:30.997 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: block_name_prefix: rbd_data.1287c927b049 2020-04-10T07:12:30.997 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: format: 2 2020-04-10T07:12:30.998 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: features: layering, exclusive-lock, journaling 2020-04-10T07:12:30.998 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: op_features: 2020-04-10T07:12:30.998 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: flags: 2020-04-10T07:12:30.998 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: create_timestamp: Fri Apr 10 07:09:58 2020 2020-04-10T07:12:30.998 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: access_timestamp: Fri Apr 10 07:09:58 2020 2020-04-10T07:12:30.998 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: modify_timestamp: Fri Apr 10 07:09:58 2020 2020-04-10T07:12:30.999 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: parent: mirror_parent/test_parent@snap 2020-04-10T07:12:30.999 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: overlap: 128 MiB 2020-04-10T07:12:30.999 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: journal: 1287c927b049 2020-04-10T07:12:30.999 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring state: enabled 2020-04-10T07:12:30.999 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring mode: snapshot 2020-04-10T07:12:30.999 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring global id: 70d86755-f73a-4bbe-9493-1b79a191cc0c 2020-04-10T07:12:31.000 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring primary: true 2020-04-10T07:12:31.001 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.002 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test_clone_v1 journal status 2020-04-10T07:12:31.002 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:31.002 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test_clone_v1 journal status 2020-04-10T07:12:31.002 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror --namespace journal status --image test_clone_v1 2020-04-10T07:12:31.081 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:minimum_set: 0 2020-04-10T07:12:31.081 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:active_set: 0 2020-04-10T07:12:31.081 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:registered clients: 2020-04-10T07:12:31.081 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: [id=, commit_position=[positions=[[object_number=1, tag_tid=1, entry_tid=1], [object_number=0, tag_tid=1, entry_tid=0]]], state=connected] 2020-04-10T07:12:31.084 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:31.084 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test_data_pool info 2020-04-10T07:12:31.085 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror --namespace info test_data_pool 2020-04-10T07:12:31.085 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.085 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test_data_pool info 2020-04-10T07:12:31.141 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:rbd image 'test_data_pool': 2020-04-10T07:12:31.141 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: size 128 MiB in 32 objects 2020-04-10T07:12:31.142 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: order 22 (4 MiB objects) 2020-04-10T07:12:31.142 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: snapshot_count: 3 2020-04-10T07:12:31.142 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: id: 12c17d32928c 2020-04-10T07:12:31.142 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: data_pool: mirror_parent 2020-04-10T07:12:31.142 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: block_name_prefix: rbd_data.3.12c17d32928c 2020-04-10T07:12:31.142 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: format: 2 2020-04-10T07:12:31.143 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: features: layering, exclusive-lock, journaling, data-pool 2020-04-10T07:12:31.143 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: op_features: 2020-04-10T07:12:31.143 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: flags: 2020-04-10T07:12:31.143 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: create_timestamp: Fri Apr 10 07:10:20 2020 2020-04-10T07:12:31.143 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: access_timestamp: Fri Apr 10 07:10:20 2020 2020-04-10T07:12:31.143 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: modify_timestamp: Fri Apr 10 07:10:20 2020 2020-04-10T07:12:31.143 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: journal: 12c17d32928c 2020-04-10T07:12:31.143 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring state: enabled 2020-04-10T07:12:31.144 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring mode: snapshot 2020-04-10T07:12:31.144 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring global id: a8183526-4331-4f0d-af4a-7c93c501e927 2020-04-10T07:12:31.144 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring primary: true 2020-04-10T07:12:31.145 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.145 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test_data_pool journal status 2020-04-10T07:12:31.146 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:31.146 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test_data_pool journal status 2020-04-10T07:12:31.146 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror --namespace journal status --image test_data_pool 2020-04-10T07:12:31.206 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:minimum_set: 0 2020-04-10T07:12:31.206 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:active_set: 0 2020-04-10T07:12:31.206 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:registered clients: 2020-04-10T07:12:31.206 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: [id=, commit_position=[positions=[[object_number=1, tag_tid=7, entry_tid=1], [object_number=0, tag_tid=7, entry_tid=0], [object_number=3, tag_tid=5, entry_tid=99], [object_number=2, tag_tid=5, entry_tid=98]]], state=connected] 2020-04-10T07:12:31.208 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:31.208 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test_force_promote info 2020-04-10T07:12:31.208 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror --namespace info test_force_promote 2020-04-10T07:12:31.209 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.209 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test_force_promote info 2020-04-10T07:12:31.255 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:rbd image 'test_force_promote': 2020-04-10T07:12:31.255 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: size 128 MiB in 32 objects 2020-04-10T07:12:31.255 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: order 22 (4 MiB objects) 2020-04-10T07:12:31.255 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: snapshot_count: 1 2020-04-10T07:12:31.255 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: id: 12381bb6482e 2020-04-10T07:12:31.255 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: block_name_prefix: rbd_data.12381bb6482e 2020-04-10T07:12:31.256 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: format: 2 2020-04-10T07:12:31.256 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: features: layering, exclusive-lock, journaling 2020-04-10T07:12:31.256 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: op_features: 2020-04-10T07:12:31.256 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: flags: 2020-04-10T07:12:31.256 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: create_timestamp: Fri Apr 10 07:09:20 2020 2020-04-10T07:12:31.256 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: access_timestamp: Fri Apr 10 07:09:20 2020 2020-04-10T07:12:31.256 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: modify_timestamp: Fri Apr 10 07:09:20 2020 2020-04-10T07:12:31.257 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: journal: 12381bb6482e 2020-04-10T07:12:31.257 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring state: enabled 2020-04-10T07:12:31.257 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring mode: snapshot 2020-04-10T07:12:31.257 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring global id: b192e243-6abc-405a-b798-b60b9d74691e 2020-04-10T07:12:31.257 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring primary: true 2020-04-10T07:12:31.258 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.258 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test_force_promote journal status 2020-04-10T07:12:31.259 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:31.259 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test_force_promote journal status 2020-04-10T07:12:31.259 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror --namespace journal status --image test_force_promote 2020-04-10T07:12:31.324 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:minimum_set: 0 2020-04-10T07:12:31.324 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:active_set: 0 2020-04-10T07:12:31.324 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:registered clients: 2020-04-10T07:12:31.325 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: [id=, commit_position=[positions=[[object_number=1, tag_tid=5, entry_tid=101], [object_number=0, tag_tid=5, entry_tid=100], [object_number=3, tag_tid=5, entry_tid=99], [object_number=2, tag_tid=5, entry_tid=98]]], state=connected] 2020-04-10T07:12:31.325 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.325 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster2 mirror rbd_mirroring omap vals 2020-04-10T07:12:31.325 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:31.325 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2 mirror rbd_mirroring omap vals 2020-04-10T07:12:31.325 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rados --cluster cluster2 -p mirror --namespace listomapvals rbd_mirroring 2020-04-10T07:12:31.360 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:global_003078e2-ed6c-4f36-8bc5-ce0c44ef25d5 2020-04-10T07:12:31.361 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (16 bytes) : 2020-04-10T07:12:31.361 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 0c 00 00 00 31 32 36 38 36 31 36 64 39 63 39 31 |....1268616d9c91| 2020-04-10T07:12:31.361 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 2020-04-10T07:12:31.361 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.361 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:global_4b1961cb-5406-443f-90f1-3627ac4ffffd 2020-04-10T07:12:31.361 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (16 bytes) : 2020-04-10T07:12:31.361 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 0c 00 00 00 31 31 38 39 64 31 39 36 39 66 31 33 |....1189d1969f13| 2020-04-10T07:12:31.362 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 2020-04-10T07:12:31.362 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.362 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:global_56419286-13e9-49f6-a20d-2abe6ede083e 2020-04-10T07:12:31.362 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (16 bytes) : 2020-04-10T07:12:31.362 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 0c 00 00 00 31 31 31 39 36 38 37 34 33 33 33 38 |....111968743338| 2020-04-10T07:12:31.362 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 2020-04-10T07:12:31.362 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.363 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:global_70d86755-f73a-4bbe-9493-1b79a191cc0c 2020-04-10T07:12:31.363 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (16 bytes) : 2020-04-10T07:12:31.363 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 0c 00 00 00 31 32 38 37 63 39 32 37 62 30 34 39 |....1287c927b049| 2020-04-10T07:12:31.363 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 2020-04-10T07:12:31.363 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.363 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:global_79225ce2-9203-4752-bc9b-35a11186acdb 2020-04-10T07:12:31.363 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (16 bytes) : 2020-04-10T07:12:31.364 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 0c 00 00 00 31 31 39 65 63 30 31 64 32 38 34 66 |....119ec01d284f| 2020-04-10T07:12:31.364 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 2020-04-10T07:12:31.364 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.364 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:global_a8183526-4331-4f0d-af4a-7c93c501e927 2020-04-10T07:12:31.364 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (16 bytes) : 2020-04-10T07:12:31.364 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 0c 00 00 00 31 32 63 31 37 64 33 32 39 32 38 63 |....12c17d32928c| 2020-04-10T07:12:31.364 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 2020-04-10T07:12:31.365 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.365 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:global_a9af7d9e-1404-4730-9e18-c0418a915346 2020-04-10T07:12:31.365 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (16 bytes) : 2020-04-10T07:12:31.365 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 0c 00 00 00 31 30 65 35 34 35 33 38 63 38 35 62 |....10e54538c85b| 2020-04-10T07:12:31.365 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 2020-04-10T07:12:31.365 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.365 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:global_b192e243-6abc-405a-b798-b60b9d74691e 2020-04-10T07:12:31.366 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (16 bytes) : 2020-04-10T07:12:31.366 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 0c 00 00 00 31 32 33 38 31 62 62 36 34 38 32 65 |....12381bb6482e| 2020-04-10T07:12:31.366 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 2020-04-10T07:12:31.366 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.367 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:global_b583e294-8b69-4e57-9966-783595c454d6 2020-04-10T07:12:31.367 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (16 bytes) : 2020-04-10T07:12:31.367 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 0c 00 00 00 31 32 65 66 66 34 61 37 36 31 39 32 |....12eff4a76192| 2020-04-10T07:12:31.367 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 2020-04-10T07:12:31.367 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.367 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image_10e54538c85b 2020-04-10T07:12:31.368 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (48 bytes) : 2020-04-10T07:12:31.368 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 02 01 2a 00 00 00 24 00 00 00 61 39 61 66 37 64 |..*...$...a9af7d| 2020-04-10T07:12:31.368 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 39 65 2d 31 34 30 34 2d 34 37 33 30 2d 39 65 31 |9e-1404-4730-9e1| 2020-04-10T07:12:31.368 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 38 2d 63 30 34 31 38 61 39 31 35 33 34 36 01 01 |8-c0418a915346..| 2020-04-10T07:12:31.368 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 2020-04-10T07:12:31.368 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.368 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image_111968743338 2020-04-10T07:12:31.369 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (48 bytes) : 2020-04-10T07:12:31.369 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 02 01 2a 00 00 00 24 00 00 00 35 36 34 31 39 32 |..*...$...564192| 2020-04-10T07:12:31.369 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 38 36 2d 31 33 65 39 2d 34 39 66 36 2d 61 32 30 |86-13e9-49f6-a20| 2020-04-10T07:12:31.369 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 64 2d 32 61 62 65 36 65 64 65 30 38 33 65 01 01 |d-2abe6ede083e..| 2020-04-10T07:12:31.369 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 2020-04-10T07:12:31.369 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.369 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image_1189d1969f13 2020-04-10T07:12:31.370 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (48 bytes) : 2020-04-10T07:12:31.370 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 02 01 2a 00 00 00 24 00 00 00 34 62 31 39 36 31 |..*...$...4b1961| 2020-04-10T07:12:31.370 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 63 62 2d 35 34 30 36 2d 34 34 33 66 2d 39 30 66 |cb-5406-443f-90f| 2020-04-10T07:12:31.370 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 31 2d 33 36 32 37 61 63 34 66 66 66 66 64 01 01 |1-3627ac4ffffd..| 2020-04-10T07:12:31.370 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 2020-04-10T07:12:31.370 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.370 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image_119ec01d284f 2020-04-10T07:12:31.371 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (48 bytes) : 2020-04-10T07:12:31.371 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 02 01 2a 00 00 00 24 00 00 00 37 39 32 32 35 63 |..*...$...79225c| 2020-04-10T07:12:31.371 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 65 32 2d 39 32 30 33 2d 34 37 35 32 2d 62 63 39 |e2-9203-4752-bc9| 2020-04-10T07:12:31.371 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 62 2d 33 35 61 31 31 31 38 36 61 63 64 62 03 01 |b-35a11186acdb..| 2020-04-10T07:12:31.371 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 2020-04-10T07:12:31.371 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.372 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image_12381bb6482e 2020-04-10T07:12:31.372 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (48 bytes) : 2020-04-10T07:12:31.372 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 02 01 2a 00 00 00 24 00 00 00 62 31 39 32 65 32 |..*...$...b192e2| 2020-04-10T07:12:31.372 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 34 33 2d 36 61 62 63 2d 34 30 35 61 2d 62 37 39 |43-6abc-405a-b79| 2020-04-10T07:12:31.372 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 38 2d 62 36 30 62 39 64 37 34 36 39 31 65 01 01 |8-b60b9d74691e..| 2020-04-10T07:12:31.372 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 2020-04-10T07:12:31.372 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.373 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image_1268616d9c91 2020-04-10T07:12:31.373 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (48 bytes) : 2020-04-10T07:12:31.373 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 02 01 2a 00 00 00 24 00 00 00 30 30 33 30 37 38 |..*...$...003078| 2020-04-10T07:12:31.373 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 65 32 2d 65 64 36 63 2d 34 66 33 36 2d 38 62 63 |e2-ed6c-4f36-8bc| 2020-04-10T07:12:31.373 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 35 2d 63 65 30 63 34 34 65 66 32 35 64 35 01 01 |5-ce0c44ef25d5..| 2020-04-10T07:12:31.373 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 2020-04-10T07:12:31.373 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.374 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image_1287c927b049 2020-04-10T07:12:31.374 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (48 bytes) : 2020-04-10T07:12:31.374 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 02 01 2a 00 00 00 24 00 00 00 37 30 64 38 36 37 |..*...$...70d867| 2020-04-10T07:12:31.374 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 35 35 2d 66 37 33 61 2d 34 62 62 65 2d 39 34 39 |55-f73a-4bbe-949| 2020-04-10T07:12:31.374 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 33 2d 31 62 37 39 61 31 39 31 63 63 30 63 01 01 |3-1b79a191cc0c..| 2020-04-10T07:12:31.374 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 2020-04-10T07:12:31.374 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.375 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image_12c17d32928c 2020-04-10T07:12:31.375 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (48 bytes) : 2020-04-10T07:12:31.375 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 02 01 2a 00 00 00 24 00 00 00 61 38 31 38 33 35 |..*...$...a81835| 2020-04-10T07:12:31.375 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 32 36 2d 34 33 33 31 2d 34 66 30 64 2d 61 66 34 |26-4331-4f0d-af4| 2020-04-10T07:12:31.375 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 61 2d 37 63 39 33 63 35 30 31 65 39 32 37 01 01 |a-7c93c501e927..| 2020-04-10T07:12:31.375 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 2020-04-10T07:12:31.375 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.375 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image_12eff4a76192 2020-04-10T07:12:31.376 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (48 bytes) : 2020-04-10T07:12:31.376 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 02 01 2a 00 00 00 24 00 00 00 62 35 38 33 65 32 |..*...$...b583e2| 2020-04-10T07:12:31.376 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 39 34 2d 38 62 36 39 2d 34 65 35 37 2d 39 39 36 |94-8b69-4e57-996| 2020-04-10T07:12:31.376 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 36 2d 37 38 33 35 39 35 63 34 35 34 64 36 01 01 |6-783595c454d6..| 2020-04-10T07:12:31.376 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 2020-04-10T07:12:31.376 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.376 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:mirror_mode 2020-04-10T07:12:31.377 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (4 bytes) : 2020-04-10T07:12:31.377 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 00 00 00 |....| 2020-04-10T07:12:31.377 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000004 2020-04-10T07:12:31.377 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.377 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:mirror_peer_90b6cfa2-5481-4edf-b462-5494aab62f6f 2020-04-10T07:12:31.377 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (136 bytes) : 2020-04-10T07:12:31.377 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 02 01 82 00 00 00 24 00 00 00 39 30 62 36 63 66 |......$...90b6cf| 2020-04-10T07:12:31.378 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 61 32 2d 35 34 38 31 2d 34 65 64 66 2d 62 34 36 |a2-5481-4edf-b46| 2020-04-10T07:12:31.378 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 32 2d 35 34 39 34 61 61 62 36 32 66 36 66 0c 00 |2-5494aab62f6f..| 2020-04-10T07:12:31.378 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 63 6c 75 73 74 65 72 31 2d 44 4e 45 0d 00 |..cluster1-DNE..| 2020-04-10T07:12:31.378 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 00 00 63 6c 69 65 6e 74 2e 6d 69 72 72 6f 72 ff |..client.mirror.| 2020-04-10T07:12:31.378 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 ff ff ff ff ff ff ff 02 24 00 00 00 36 32 30 64 |........$...620d| 2020-04-10T07:12:31.378 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000060 64 62 30 30 2d 30 66 34 39 2d 34 37 35 38 2d 61 |db00-0f49-4758-a| 2020-04-10T07:12:31.379 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000070 32 31 36 2d 38 64 39 63 63 38 39 30 64 30 66 61 |216-8d9cc890d0fa| 2020-04-10T07:12:31.379 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000080 0d 1c 90 5e 25 bc c2 1c |...^%...| 2020-04-10T07:12:31.379 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000088 2020-04-10T07:12:31.379 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.379 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:mirror_uuid 2020-04-10T07:12:31.379 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (36 bytes) : 2020-04-10T07:12:31.379 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 38 32 35 33 66 64 65 32 2d 38 36 66 38 2d 34 34 |8253fde2-86f8-44| 2020-04-10T07:12:31.380 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 33 61 2d 62 34 62 66 2d 65 63 37 64 61 65 32 66 |3a-b4bf-ec7dae2f| 2020-04-10T07:12:31.380 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 61 66 61 33 |afa3| 2020-04-10T07:12:31.380 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000024 2020-04-10T07:12:31.380 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.380 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:remote_status_global_003078e2-ed6c-4f36-8bc5-ce0c44ef25d5_620ddb00-0f49-4758-a216-8d9cc890d0fa 2020-04-10T07:12:31.380 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (77 bytes) : 2020-04-10T07:12:31.380 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 24 11 00 00 00 00 00 00 01 |..,....$........| 2020-04-10T07:12:31.381 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 b6 31 b9 ca 10 00 |...........1....| 2020-04-10T07:12:31.381 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T07:12:31.381 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 15 00 00 00 06 07 00 00 00 73 74 6f |.............sto| 2020-04-10T07:12:31.381 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 70 70 65 64 12 1c 90 5e f2 97 d8 17 00 |pped...^.....| 2020-04-10T07:12:31.381 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000004d 2020-04-10T07:12:31.381 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.381 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:remote_status_global_3876de2e-b17d-4c86-94c4-90a11916f44e_620ddb00-0f49-4758-a216-8d9cc890d0fa 2020-04-10T07:12:31.382 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (96 bytes) : 2020-04-10T07:12:31.382 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 28 11 00 00 00 00 00 00 01 |..,....(........| 2020-04-10T07:12:31.382 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 1b 92 fa 21 10 00 |.............!..| 2020-04-10T07:12:31.382 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T07:12:31.382 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 28 00 00 00 01 1a 00 00 00 65 72 72 |....(........err| 2020-04-10T07:12:31.383 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 6f 72 20 62 6f 6f 74 73 74 72 61 70 70 69 6e 67 |or bootstrapping| 2020-04-10T07:12:31.383 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 20 72 65 70 6c 61 79 14 1c 90 5e 71 cc 13 21 00 | replay...^q..!.| 2020-04-10T07:12:31.383 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000060 2020-04-10T07:12:31.383 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.383 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:remote_status_global_4334771a-4312-4fab-8e12-36d78acc7af1_620ddb00-0f49-4758-a216-8d9cc890d0fa 2020-04-10T07:12:31.383 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (96 bytes) : 2020-04-10T07:12:31.383 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 2a 11 00 00 00 00 00 00 01 |..,....*........| 2020-04-10T07:12:31.384 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 ed ce d6 f8 10 00 |................| 2020-04-10T07:12:31.384 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T07:12:31.384 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 28 00 00 00 01 1a 00 00 00 65 72 72 |....(........err| 2020-04-10T07:12:31.384 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 6f 72 20 62 6f 6f 74 73 74 72 61 70 70 69 6e 67 |or bootstrapping| 2020-04-10T07:12:31.384 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 20 72 65 70 6c 61 79 14 1c 90 5e fc 5d 37 21 00 | replay...^.]7!.| 2020-04-10T07:12:31.384 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000060 2020-04-10T07:12:31.384 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.385 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:remote_status_global_4b1961cb-5406-443f-90f1-3627ac4ffffd_620ddb00-0f49-4758-a216-8d9cc890d0fa 2020-04-10T07:12:31.385 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (92 bytes) : 2020-04-10T07:12:31.385 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 26 11 00 00 00 00 00 00 01 |..,....&........| 2020-04-10T07:12:31.385 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 d5 a9 fe 0b 10 00 |................| 2020-04-10T07:12:31.385 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T07:12:31.385 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 24 00 00 00 06 16 00 00 00 6c 6f 63 |....$........loc| 2020-04-10T07:12:31.385 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 61 6c 20 69 6d 61 67 65 20 69 73 20 70 72 69 6d |al image is prim| 2020-04-10T07:12:31.386 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 61 72 79 13 1c 90 5e 92 fa f6 1a 00 |ary...^.....| 2020-04-10T07:12:31.386 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000005c 2020-04-10T07:12:31.386 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.386 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:remote_status_global_56419286-13e9-49f6-a20d-2abe6ede083e_620ddb00-0f49-4758-a216-8d9cc890d0fa 2020-04-10T07:12:31.386 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (77 bytes) : 2020-04-10T07:12:31.386 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 24 11 00 00 00 00 00 00 01 |..,....$........| 2020-04-10T07:12:31.386 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 b6 31 b9 ca 10 00 |...........1....| 2020-04-10T07:12:31.387 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T07:12:31.387 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 15 00 00 00 06 07 00 00 00 73 74 6f |.............sto| 2020-04-10T07:12:31.387 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 70 70 65 64 12 1c 90 5e d3 cb 8d 17 00 |pped...^.....| 2020-04-10T07:12:31.387 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000004d 2020-04-10T07:12:31.387 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.387 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:remote_status_global_5d3e94e2-1c6c-4146-b461-67ecadf338c4_620ddb00-0f49-4758-a216-8d9cc890d0fa 2020-04-10T07:12:31.387 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (96 bytes) : 2020-04-10T07:12:31.388 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 2a 11 00 00 00 00 00 00 01 |..,....*........| 2020-04-10T07:12:31.388 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 ed ce d6 f8 10 00 |................| 2020-04-10T07:12:31.388 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T07:12:31.388 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 28 00 00 00 01 1a 00 00 00 65 72 72 |....(........err| 2020-04-10T07:12:31.388 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 6f 72 20 62 6f 6f 74 73 74 72 61 70 70 69 6e 67 |or bootstrapping| 2020-04-10T07:12:31.388 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 20 72 65 70 6c 61 79 14 1c 90 5e 7d be 5d 21 00 | replay...^}.]!.| 2020-04-10T07:12:31.388 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000060 2020-04-10T07:12:31.389 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.389 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:remote_status_global_687e9f7d-ae46-4324-b8f7-7c16605e0fa6_620ddb00-0f49-4758-a216-8d9cc890d0fa 2020-04-10T07:12:31.389 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (96 bytes) : 2020-04-10T07:12:31.389 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 26 11 00 00 00 00 00 00 01 |..,....&........| 2020-04-10T07:12:31.389 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 d5 a9 fe 0b 10 00 |................| 2020-04-10T07:12:31.389 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T07:12:31.389 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 28 00 00 00 01 1a 00 00 00 65 72 72 |....(........err| 2020-04-10T07:12:31.390 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 6f 72 20 62 6f 6f 74 73 74 72 61 70 70 69 6e 67 |or bootstrapping| 2020-04-10T07:12:31.390 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 20 72 65 70 6c 61 79 13 1c 90 5e 59 4a f6 19 00 | replay...^YJ...| 2020-04-10T07:12:31.390 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000060 2020-04-10T07:12:31.390 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.390 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:remote_status_global_697ea219-b252-4c33-ba20-9b31ea5be82a_620ddb00-0f49-4758-a216-8d9cc890d0fa 2020-04-10T07:12:31.390 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (96 bytes) : 2020-04-10T07:12:31.390 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 24 11 00 00 00 00 00 00 01 |..,....$........| 2020-04-10T07:12:31.391 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 b6 31 b9 ca 10 00 |...........1....| 2020-04-10T07:12:31.391 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T07:12:31.391 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 28 00 00 00 01 1a 00 00 00 65 72 72 |....(........err| 2020-04-10T07:12:31.391 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 6f 72 20 62 6f 6f 74 73 74 72 61 70 70 69 6e 67 |or bootstrapping| 2020-04-10T07:12:31.391 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 20 72 65 70 6c 61 79 0d 1c 90 5e c4 f6 57 16 00 | replay...^..W..| 2020-04-10T07:12:31.391 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000060 2020-04-10T07:12:31.391 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.391 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:remote_status_global_70d86755-f73a-4bbe-9493-1b79a191cc0c_620ddb00-0f49-4758-a216-8d9cc890d0fa 2020-04-10T07:12:31.392 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (77 bytes) : 2020-04-10T07:12:31.392 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 28 11 00 00 00 00 00 00 01 |..,....(........| 2020-04-10T07:12:31.392 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 1b 92 fa 21 10 00 |.............!..| 2020-04-10T07:12:31.392 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T07:12:31.392 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 15 00 00 00 06 07 00 00 00 73 74 6f |.............sto| 2020-04-10T07:12:31.392 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 70 70 65 64 14 1c 90 5e 65 0d de 26 00 |pped...^e..&.| 2020-04-10T07:12:31.392 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000004d 2020-04-10T07:12:31.393 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.393 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:remote_status_global_79225ce2-9203-4752-bc9b-35a11186acdb_620ddb00-0f49-4758-a216-8d9cc890d0fa 2020-04-10T07:12:31.393 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (97 bytes) : 2020-04-10T07:12:31.393 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 28 11 00 00 00 00 00 00 01 |..,....(........| 2020-04-10T07:12:31.393 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 1b 92 fa 21 10 00 |.............!..| 2020-04-10T07:12:31.393 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T07:12:31.393 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 29 00 00 00 00 1b 00 00 00 72 65 6d |....)........rem| 2020-04-10T07:12:31.394 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 6f 74 65 20 69 6d 61 67 65 20 69 73 20 6e 6f 6e |ote image is non| 2020-04-10T07:12:31.394 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 2d 70 72 69 6d 61 72 79 14 1c 90 5e 84 ba 8e 21 |-primary...^...!| 2020-04-10T07:12:31.394 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000060 00 |.| 2020-04-10T07:12:31.394 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000061 2020-04-10T07:12:31.394 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.394 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:remote_status_global_925c4d02-9086-4edd-b4f3-79c1b7e3734c_620ddb00-0f49-4758-a216-8d9cc890d0fa 2020-04-10T07:12:31.394 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (96 bytes) : 2020-04-10T07:12:31.395 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 26 11 00 00 00 00 00 00 01 |..,....&........| 2020-04-10T07:12:31.395 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 d5 a9 fe 0b 10 00 |................| 2020-04-10T07:12:31.395 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T07:12:31.395 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 28 00 00 00 01 1a 00 00 00 65 72 72 |....(........err| 2020-04-10T07:12:31.395 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 6f 72 20 62 6f 6f 74 73 74 72 61 70 70 69 6e 67 |or bootstrapping| 2020-04-10T07:12:31.396 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 20 72 65 70 6c 61 79 13 1c 90 5e e5 b1 11 1a 00 | replay...^.....| 2020-04-10T07:12:31.396 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000060 2020-04-10T07:12:31.396 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.396 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:remote_status_global_a8183526-4331-4f0d-af4a-7c93c501e927_620ddb00-0f49-4758-a216-8d9cc890d0fa 2020-04-10T07:12:31.396 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (77 bytes) : 2020-04-10T07:12:31.396 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 24 11 00 00 00 00 00 00 01 |..,....$........| 2020-04-10T07:12:31.396 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 b6 31 b9 ca 10 00 |...........1....| 2020-04-10T07:12:31.397 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T07:12:31.397 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 15 00 00 00 06 07 00 00 00 73 74 6f |.............sto| 2020-04-10T07:12:31.397 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 70 70 65 64 12 1c 90 5e 24 58 f6 17 00 |pped...^$X...| 2020-04-10T07:12:31.397 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000004d 2020-04-10T07:12:31.397 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.397 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:remote_status_global_a9af7d9e-1404-4730-9e18-c0418a915346_620ddb00-0f49-4758-a216-8d9cc890d0fa 2020-04-10T07:12:31.397 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (77 bytes) : 2020-04-10T07:12:31.397 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 28 11 00 00 00 00 00 00 01 |..,....(........| 2020-04-10T07:12:31.398 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 1b 92 fa 21 10 00 |.............!..| 2020-04-10T07:12:31.398 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T07:12:31.398 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 15 00 00 00 06 07 00 00 00 73 74 6f |.............sto| 2020-04-10T07:12:31.398 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 70 70 65 64 14 1c 90 5e 6d 07 6e 26 00 |pped...^m.n&.| 2020-04-10T07:12:31.398 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000004d 2020-04-10T07:12:31.398 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.398 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:remote_status_global_b192e243-6abc-405a-b798-b60b9d74691e_620ddb00-0f49-4758-a216-8d9cc890d0fa 2020-04-10T07:12:31.399 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (92 bytes) : 2020-04-10T07:12:31.399 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 2a 11 00 00 00 00 00 00 01 |..,....*........| 2020-04-10T07:12:31.399 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 ed ce d6 f8 10 00 |................| 2020-04-10T07:12:31.399 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T07:12:31.399 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 24 00 00 00 06 16 00 00 00 6c 6f 63 |....$........loc| 2020-04-10T07:12:31.399 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 61 6c 20 69 6d 61 67 65 20 69 73 20 70 72 69 6d |al image is prim| 2020-04-10T07:12:31.399 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 61 72 79 14 1c 90 5e 69 1c de 21 00 |ary...^i..!.| 2020-04-10T07:12:31.400 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000005c 2020-04-10T07:12:31.400 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.400 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:remote_status_global_b583e294-8b69-4e57-9966-783595c454d6_620ddb00-0f49-4758-a216-8d9cc890d0fa 2020-04-10T07:12:31.400 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (77 bytes) : 2020-04-10T07:12:31.400 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 26 11 00 00 00 00 00 00 01 |..,....&........| 2020-04-10T07:12:31.400 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 d5 a9 fe 0b 10 00 |................| 2020-04-10T07:12:31.401 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T07:12:31.401 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 15 00 00 00 06 07 00 00 00 73 74 6f |.............sto| 2020-04-10T07:12:31.401 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 70 70 65 64 13 1c 90 5e 61 9b 79 1f 00 |pped...^a.y..| 2020-04-10T07:12:31.401 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000004d 2020-04-10T07:12:31.401 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.401 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:status_global_003078e2-ed6c-4f36-8bc5-ce0c44ef25d5 2020-04-10T07:12:31.402 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (92 bytes) : 2020-04-10T07:12:31.402 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 9e 11 00 00 00 00 00 00 01 |..,.............| 2020-04-10T07:12:31.402 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 74 d8 db 3f 10 00 |..........t..?..| 2020-04-10T07:12:31.402 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T07:12:31.402 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 24 00 00 00 06 16 00 00 00 6c 6f 63 |....$........loc| 2020-04-10T07:12:31.402 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 61 6c 20 69 6d 61 67 65 20 69 73 20 70 72 69 6d |al image is prim| 2020-04-10T07:12:31.402 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 61 72 79 5f 1c 90 5e 7e bf 4a 0b 00 |ary_..^~.J..| 2020-04-10T07:12:31.403 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000005c 2020-04-10T07:12:31.403 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.403 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:status_global_3876de2e-b17d-4c86-94c4-90a11916f44e 2020-04-10T07:12:31.403 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (96 bytes) : 2020-04-10T07:12:31.403 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 9e 11 00 00 00 00 00 00 01 |..,.............| 2020-04-10T07:12:31.403 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 74 d8 db 3f 10 00 |..........t..?..| 2020-04-10T07:12:31.403 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T07:12:31.404 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 28 00 00 00 01 1a 00 00 00 65 72 72 |....(........err| 2020-04-10T07:12:31.404 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 6f 72 20 62 6f 6f 74 73 74 72 61 70 70 69 6e 67 |or bootstrapping| 2020-04-10T07:12:31.404 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 20 72 65 70 6c 61 79 39 1c 90 5e 66 bb b0 03 00 | replay9..^f....| 2020-04-10T07:12:31.404 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000060 2020-04-10T07:12:31.404 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.404 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:status_global_4334771a-4312-4fab-8e12-36d78acc7af1 2020-04-10T07:12:31.404 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (96 bytes) : 2020-04-10T07:12:31.405 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 96 11 00 00 00 00 00 00 01 |..,.............| 2020-04-10T07:12:31.405 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 3e 69 14 eb 10 00 |..........>i....| 2020-04-10T07:12:31.405 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T07:12:31.405 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 28 00 00 00 01 1a 00 00 00 65 72 72 |....(........err| 2020-04-10T07:12:31.405 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 6f 72 20 62 6f 6f 74 73 74 72 61 70 70 69 6e 67 |or bootstrapping| 2020-04-10T07:12:31.405 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 20 72 65 70 6c 61 79 18 1c 90 5e 46 4c d8 39 00 | replay...^FL.9.| 2020-04-10T07:12:31.405 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000060 2020-04-10T07:12:31.406 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.406 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:status_global_4b1961cb-5406-443f-90f1-3627ac4ffffd 2020-04-10T07:12:31.406 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (186 bytes) : 2020-04-10T07:12:31.406 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 96 11 00 00 00 00 00 00 01 |..,.............| 2020-04-10T07:12:31.406 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 3e 69 14 eb 10 00 |..........>i....| 2020-04-10T07:12:31.406 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T07:12:31.406 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 82 00 00 00 04 74 00 00 00 72 65 70 |.........t...rep| 2020-04-10T07:12:31.407 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 6c 61 79 69 6e 67 2c 20 7b 22 72 65 70 6c 61 79 |laying, {"replay| 2020-04-10T07:12:31.407 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 5f 73 74 61 74 65 22 3a 20 22 69 64 6c 65 22 2c |_state": "idle",| 2020-04-10T07:12:31.407 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000060 20 22 72 65 6d 6f 74 65 5f 73 6e 61 70 73 68 6f | "remote_snapsho| 2020-04-10T07:12:31.407 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000070 74 5f 74 69 6d 65 73 74 61 6d 70 22 3a 20 31 35 |t_timestamp": 15| 2020-04-10T07:12:31.407 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000080 38 36 35 30 32 35 39 38 2c 20 22 6c 6f 63 61 6c |86502598, "local| 2020-04-10T07:12:31.407 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000090 5f 73 6e 61 70 73 68 6f 74 5f 74 69 6d 65 73 74 |_snapshot_timest| 2020-04-10T07:12:31.407 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:000000a0 61 6d 70 22 3a 20 31 35 38 36 35 30 32 35 39 38 |amp": 1586502598| 2020-04-10T07:12:31.408 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:000000b0 7d 46 1c 90 5e fc 4a 31 09 00 |}F..^.J1..| 2020-04-10T07:12:31.408 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:000000ba 2020-04-10T07:12:31.408 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.408 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:status_global_56419286-13e9-49f6-a20d-2abe6ede083e 2020-04-10T07:12:31.408 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (92 bytes) : 2020-04-10T07:12:31.408 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 9e 11 00 00 00 00 00 00 01 |..,.............| 2020-04-10T07:12:31.408 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 74 d8 db 3f 10 00 |..........t..?..| 2020-04-10T07:12:31.409 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T07:12:31.409 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 24 00 00 00 06 16 00 00 00 6c 6f 63 |....$........loc| 2020-04-10T07:12:31.409 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 61 6c 20 69 6d 61 67 65 20 69 73 20 70 72 69 6d |al image is prim| 2020-04-10T07:12:31.409 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 61 72 79 5f 1c 90 5e ad 9a 68 0b 00 |ary_..^..h..| 2020-04-10T07:12:31.409 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000005c 2020-04-10T07:12:31.409 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.409 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:status_global_5d3e94e2-1c6c-4146-b461-67ecadf338c4 2020-04-10T07:12:31.410 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (96 bytes) : 2020-04-10T07:12:31.410 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 96 11 00 00 00 00 00 00 01 |..,.............| 2020-04-10T07:12:31.410 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 3e 69 14 eb 10 00 |..........>i....| 2020-04-10T07:12:31.410 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T07:12:31.411 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 28 00 00 00 01 1a 00 00 00 65 72 72 |....(........err| 2020-04-10T07:12:31.411 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 6f 72 20 62 6f 6f 74 73 74 72 61 70 70 69 6e 67 |or bootstrapping| 2020-04-10T07:12:31.411 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 20 72 65 70 6c 61 79 18 1c 90 5e 2d da d8 39 00 | replay...^-..9.| 2020-04-10T07:12:31.411 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000060 2020-04-10T07:12:31.411 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.412 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:status_global_697ea219-b252-4c33-ba20-9b31ea5be82a 2020-04-10T07:12:31.412 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (96 bytes) : 2020-04-10T07:12:31.412 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 9e 11 00 00 00 00 00 00 01 |..,.............| 2020-04-10T07:12:31.412 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 74 d8 db 3f 10 00 |..........t..?..| 2020-04-10T07:12:31.412 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T07:12:31.412 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 28 00 00 00 01 1a 00 00 00 65 72 72 |....(........err| 2020-04-10T07:12:31.412 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 6f 72 20 62 6f 6f 74 73 74 72 61 70 70 69 6e 67 |or bootstrapping| 2020-04-10T07:12:31.413 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 20 72 65 70 6c 61 79 18 1c 90 5e 4f a5 27 3a 00 | replay...^O.':.| 2020-04-10T07:12:31.413 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000060 2020-04-10T07:12:31.413 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.413 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:status_global_70d86755-f73a-4bbe-9493-1b79a191cc0c 2020-04-10T07:12:31.413 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (92 bytes) : 2020-04-10T07:12:31.413 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 96 11 00 00 00 00 00 00 01 |..,.............| 2020-04-10T07:12:31.413 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 3e 69 14 eb 10 00 |..........>i....| 2020-04-10T07:12:31.414 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T07:12:31.414 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 24 00 00 00 06 16 00 00 00 6c 6f 63 |....$........loc| 2020-04-10T07:12:31.414 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 61 6c 20 69 6d 61 67 65 20 69 73 20 70 72 69 6d |al image is prim| 2020-04-10T07:12:31.414 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 61 72 79 5f 1c 90 5e 3b 72 2a 0b 00 |ary_..^;r*..| 2020-04-10T07:12:31.414 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000005c 2020-04-10T07:12:31.415 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.415 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:status_global_79225ce2-9203-4752-bc9b-35a11186acdb 2020-04-10T07:12:31.415 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (96 bytes) : 2020-04-10T07:12:31.415 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 9e 11 00 00 00 00 00 00 01 |..,.............| 2020-04-10T07:12:31.415 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 74 d8 db 3f 10 00 |..........t..?..| 2020-04-10T07:12:31.415 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T07:12:31.415 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 28 00 00 00 01 1a 00 00 00 65 72 72 |....(........err| 2020-04-10T07:12:31.415 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 6f 72 20 62 6f 6f 74 73 74 72 61 70 70 69 6e 67 |or bootstrapping| 2020-04-10T07:12:31.416 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 20 72 65 70 6c 61 79 5f 1c 90 5e 2a 50 10 0b 00 | replay_..^*P...| 2020-04-10T07:12:31.416 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000060 2020-04-10T07:12:31.416 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.416 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:status_global_925c4d02-9086-4edd-b4f3-79c1b7e3734c 2020-04-10T07:12:31.416 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (96 bytes) : 2020-04-10T07:12:31.416 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 96 11 00 00 00 00 00 00 01 |..,.............| 2020-04-10T07:12:31.417 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 3e 69 14 eb 10 00 |..........>i....| 2020-04-10T07:12:31.417 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T07:12:31.417 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 28 00 00 00 01 1a 00 00 00 65 72 72 |....(........err| 2020-04-10T07:12:31.417 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 6f 72 20 62 6f 6f 74 73 74 72 61 70 70 69 6e 67 |or bootstrapping| 2020-04-10T07:12:31.417 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 20 72 65 70 6c 61 79 39 1c 90 5e 8e 30 f9 03 00 | replay9..^.0...| 2020-04-10T07:12:31.417 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000060 2020-04-10T07:12:31.417 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.417 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:status_global_a8183526-4331-4f0d-af4a-7c93c501e927 2020-04-10T07:12:31.418 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (92 bytes) : 2020-04-10T07:12:31.418 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 9e 11 00 00 00 00 00 00 01 |..,.............| 2020-04-10T07:12:31.418 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 74 d8 db 3f 10 00 |..........t..?..| 2020-04-10T07:12:31.418 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T07:12:31.418 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 24 00 00 00 06 16 00 00 00 6c 6f 63 |....$........loc| 2020-04-10T07:12:31.418 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 61 6c 20 69 6d 61 67 65 20 69 73 20 70 72 69 6d |al image is prim| 2020-04-10T07:12:31.418 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 61 72 79 5f 1c 90 5e 01 30 a8 0b 00 |ary_..^.0...| 2020-04-10T07:12:31.419 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000005c 2020-04-10T07:12:31.419 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.419 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:status_global_a9af7d9e-1404-4730-9e18-c0418a915346 2020-04-10T07:12:31.419 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (92 bytes) : 2020-04-10T07:12:31.419 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 9e 11 00 00 00 00 00 00 01 |..,.............| 2020-04-10T07:12:31.419 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 74 d8 db 3f 10 00 |..........t..?..| 2020-04-10T07:12:31.419 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T07:12:31.420 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 24 00 00 00 06 16 00 00 00 6c 6f 63 |....$........loc| 2020-04-10T07:12:31.420 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 61 6c 20 69 6d 61 67 65 20 69 73 20 70 72 69 6d |al image is prim| 2020-04-10T07:12:31.420 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 61 72 79 5f 1c 90 5e 49 98 c6 0b 00 |ary_..^I....| 2020-04-10T07:12:31.420 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000005c 2020-04-10T07:12:31.420 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.420 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:status_global_b192e243-6abc-405a-b798-b60b9d74691e 2020-04-10T07:12:31.421 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (92 bytes) : 2020-04-10T07:12:31.421 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 96 11 00 00 00 00 00 00 01 |..,.............| 2020-04-10T07:12:31.421 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 3e 69 14 eb 10 00 |..........>i....| 2020-04-10T07:12:31.421 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T07:12:31.421 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 24 00 00 00 06 16 00 00 00 6c 6f 63 |....$........loc| 2020-04-10T07:12:31.421 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 61 6c 20 69 6d 61 67 65 20 69 73 20 70 72 69 6d |al image is prim| 2020-04-10T07:12:31.422 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 61 72 79 5f 1c 90 5e dd 76 ec 0a 00 |ary_..^.v...| 2020-04-10T07:12:31.422 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000005c 2020-04-10T07:12:31.422 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.422 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:status_global_b583e294-8b69-4e57-9966-783595c454d6 2020-04-10T07:12:31.422 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (92 bytes) : 2020-04-10T07:12:31.422 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 96 11 00 00 00 00 00 00 01 |..,.............| 2020-04-10T07:12:31.422 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 3e 69 14 eb 10 00 |..........>i....| 2020-04-10T07:12:31.423 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T07:12:31.423 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 24 00 00 00 06 16 00 00 00 6c 6f 63 |....$........loc| 2020-04-10T07:12:31.423 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 61 6c 20 69 6d 61 67 65 20 69 73 20 70 72 69 6d |al image is prim| 2020-04-10T07:12:31.423 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 61 72 79 5f 1c 90 5e 7b f1 2a 0b 00 |ary_..^{.*..| 2020-04-10T07:12:31.423 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000005c 2020-04-10T07:12:31.423 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.423 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.424 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster2 mirror ns1 images 2020-04-10T07:12:31.424 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:31.424 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2 mirror ns1 images 2020-04-10T07:12:31.424 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror --namespace ns1 ls -l 2020-04-10T07:12:31.425 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:31.425 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2 mirrorns1 mirror pool info 2020-04-10T07:12:31.425 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror --namespace ns1 mirror pool info 2020-04-10T07:12:31.425 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.426 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster2 mirrorns1 mirror pool info 2020-04-10T07:12:31.438 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Mode: image 2020-04-10T07:12:31.442 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:31.442 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2 mirrorns1 mirror pool status 2020-04-10T07:12:31.443 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster2 -p mirror --namespace ns1 mirror pool status --verbose 2020-04-10T07:12:31.443 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.443 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster2 mirrorns1 mirror pool status 2020-04-10T07:12:31.485 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:health: OK 2020-04-10T07:12:31.485 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:daemon health: OK 2020-04-10T07:12:31.485 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image health: OK 2020-04-10T07:12:31.485 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:images: 0 total 2020-04-10T07:12:31.485 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.486 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:DAEMONS 2020-04-10T07:12:31.486 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:service 4491: 2020-04-10T07:12:31.486 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: instance_id: 4502 2020-04-10T07:12:31.486 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: client_id: mirror.2 2020-04-10T07:12:31.486 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: hostname: smithi204 2020-04-10T07:12:31.486 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: version: 15.2.0-148-g39357d2 2020-04-10T07:12:31.486 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: leader: false 2020-04-10T07:12:31.487 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: health: OK 2020-04-10T07:12:31.487 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.487 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:service 4499: 2020-04-10T07:12:31.487 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: instance_id: 4510 2020-04-10T07:12:31.487 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: client_id: mirror.3 2020-04-10T07:12:31.487 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: hostname: smithi204 2020-04-10T07:12:31.487 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: version: 15.2.0-148-g39357d2 2020-04-10T07:12:31.488 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: leader: true 2020-04-10T07:12:31.488 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: health: OK 2020-04-10T07:12:31.488 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.488 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.493 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:IMAGES 2020-04-10T07:12:31.493 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:31.493 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror --namespace ns1 ls 2020-04-10T07:12:31.532 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster2 mirror ns1 rbd_mirroring omap vals 2020-04-10T07:12:31.532 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2 mirror ns1 rbd_mirroring omap vals 2020-04-10T07:12:31.532 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rados --cluster cluster2 -p mirror --namespace ns1 listomapvals rbd_mirroring 2020-04-10T07:12:31.566 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:mirror_mode 2020-04-10T07:12:31.566 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (4 bytes) : 2020-04-10T07:12:31.566 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 00 00 00 |....| 2020-04-10T07:12:31.566 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000004 2020-04-10T07:12:31.567 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.567 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:mirror_uuid 2020-04-10T07:12:31.567 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (36 bytes) : 2020-04-10T07:12:31.567 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 36 64 34 64 30 63 65 38 2d 36 39 36 35 2d 34 35 |6d4d0ce8-6965-45| 2020-04-10T07:12:31.567 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 63 32 2d 38 62 64 66 2d 38 38 38 64 36 39 63 39 |c2-8bdf-888d69c9| 2020-04-10T07:12:31.567 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 30 61 35 34 |0a54| 2020-04-10T07:12:31.568 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000024 2020-04-10T07:12:31.568 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.571 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:31.571 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2 mirror ns2 images 2020-04-10T07:12:31.571 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror --namespace ns2 ls -l 2020-04-10T07:12:31.571 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.571 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster2 mirror ns2 images 2020-04-10T07:12:31.607 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.608 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster2 mirrorns2 mirror pool info 2020-04-10T07:12:31.608 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:31.608 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2 mirrorns2 mirror pool info 2020-04-10T07:12:31.608 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror --namespace ns2 mirror pool info 2020-04-10T07:12:31.648 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Mode: image 2020-04-10T07:12:31.652 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:31.652 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2 mirrorns2 mirror pool status 2020-04-10T07:12:31.652 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster2 -p mirror --namespace ns2 mirror pool status --verbose 2020-04-10T07:12:31.652 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.652 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster2 mirrorns2 mirror pool status 2020-04-10T07:12:31.695 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:health: OK 2020-04-10T07:12:31.695 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:daemon health: OK 2020-04-10T07:12:31.696 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image health: OK 2020-04-10T07:12:31.696 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:images: 0 total 2020-04-10T07:12:31.696 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.696 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:DAEMONS 2020-04-10T07:12:31.697 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:service 4491: 2020-04-10T07:12:31.697 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: instance_id: 4502 2020-04-10T07:12:31.697 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: client_id: mirror.2 2020-04-10T07:12:31.697 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: hostname: smithi204 2020-04-10T07:12:31.697 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: version: 15.2.0-148-g39357d2 2020-04-10T07:12:31.698 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: leader: false 2020-04-10T07:12:31.698 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: health: OK 2020-04-10T07:12:31.698 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.698 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:service 4499: 2020-04-10T07:12:31.699 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: instance_id: 4510 2020-04-10T07:12:31.699 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: client_id: mirror.3 2020-04-10T07:12:31.699 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: hostname: smithi204 2020-04-10T07:12:31.699 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: version: 15.2.0-148-g39357d2 2020-04-10T07:12:31.699 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: leader: true 2020-04-10T07:12:31.700 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: health: OK 2020-04-10T07:12:31.700 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.700 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.703 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:31.703 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror --namespace ns2 ls 2020-04-10T07:12:31.704 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:IMAGES 2020-04-10T07:12:31.745 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2 mirror ns2 rbd_mirroring omap vals 2020-04-10T07:12:31.745 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rados --cluster cluster2 -p mirror --namespace ns2 listomapvals rbd_mirroring 2020-04-10T07:12:31.745 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster2 mirror ns2 rbd_mirroring omap vals 2020-04-10T07:12:31.782 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:mirror_mode 2020-04-10T07:12:31.782 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (4 bytes) : 2020-04-10T07:12:31.782 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 00 00 00 |....| 2020-04-10T07:12:31.782 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000004 2020-04-10T07:12:31.783 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.783 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:mirror_uuid 2020-04-10T07:12:31.783 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (36 bytes) : 2020-04-10T07:12:31.783 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 65 64 32 35 38 30 38 39 2d 34 36 39 34 2d 34 61 |ed258089-4694-4a| 2020-04-10T07:12:31.783 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 62 63 2d 39 65 32 35 2d 63 34 31 62 37 62 62 34 |bc-9e25-c41b7bb4| 2020-04-10T07:12:31.783 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 64 32 39 65 |d29e| 2020-04-10T07:12:31.783 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000024 2020-04-10T07:12:31.784 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.784 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.785 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster2 mirror_parent images 2020-04-10T07:12:31.785 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:31.785 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2 mirror_parent images 2020-04-10T07:12:31.785 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror_parent --namespace ls -l 2020-04-10T07:12:31.842 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:NAME SIZE PARENT FMT PROT LOCK 2020-04-10T07:12:31.843 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test_parent 128 MiB 2 2020-04-10T07:12:31.843 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test_parent@snap 128 MiB 2 yes 2020-04-10T07:12:31.846 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:31.846 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2 mirror_parent mirror pool info 2020-04-10T07:12:31.847 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror_parent --namespace mirror pool info 2020-04-10T07:12:31.847 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.847 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster2 mirror_parent mirror pool info 2020-04-10T07:12:31.886 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Mode: image 2020-04-10T07:12:31.886 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Site Name: cluster2-DNE 2020-04-10T07:12:31.886 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.886 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Peer Sites: 2020-04-10T07:12:31.886 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.886 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:UUID: 39a2eac0-9f81-4c3b-a329-98e40ae52daf 2020-04-10T07:12:31.887 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Name: cluster1-DNE 2020-04-10T07:12:31.887 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Mirror UUID: fa567d93-f465-4764-93c7-129d993cf8a1 2020-04-10T07:12:31.887 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Direction: rx-tx 2020-04-10T07:12:31.887 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Client: client.mirror 2020-04-10T07:12:31.888 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:31.889 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2 mirror_parent mirror pool status 2020-04-10T07:12:31.889 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster2 -p mirror_parent --namespace mirror pool status --verbose 2020-04-10T07:12:31.889 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.889 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster2 mirror_parent mirror pool status 2020-04-10T07:12:31.929 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:health: WARNING 2020-04-10T07:12:31.929 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:daemon health: OK 2020-04-10T07:12:31.929 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image health: WARNING 2020-04-10T07:12:31.929 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:images: 1 total 2020-04-10T07:12:31.929 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 1 unknown 2020-04-10T07:12:31.930 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.930 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:DAEMONS 2020-04-10T07:12:31.930 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:service 4491: 2020-04-10T07:12:31.930 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: instance_id: 2020-04-10T07:12:31.930 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: client_id: mirror.2 2020-04-10T07:12:31.930 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: hostname: smithi204 2020-04-10T07:12:31.930 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: version: 15.2.0-148-g39357d2 2020-04-10T07:12:31.931 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: leader: false 2020-04-10T07:12:31.931 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: health: OK 2020-04-10T07:12:31.931 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.931 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:service 4499: 2020-04-10T07:12:31.931 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: instance_id: 4522 2020-04-10T07:12:31.931 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: client_id: mirror.3 2020-04-10T07:12:31.932 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: hostname: smithi204 2020-04-10T07:12:31.932 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: version: 15.2.0-148-g39357d2 2020-04-10T07:12:31.932 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: leader: true 2020-04-10T07:12:31.932 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: health: OK 2020-04-10T07:12:31.932 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.932 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:31.963 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:IMAGES 2020-04-10T07:12:31.964 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test_parent: 2020-04-10T07:12:31.964 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: global_id: dd523e97-79dd-49a7-b2c9-df255e9660d4 2020-04-10T07:12:31.964 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+stopped 2020-04-10T07:12:31.964 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: local image is primary 2020-04-10T07:12:31.964 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: service: mirror.3 on smithi204 2020-04-10T07:12:31.964 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 07:12:31 2020-04-10T07:12:31.965 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: peer_sites: 2020-04-10T07:12:31.965 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: name: cluster1-DNE 2020-04-10T07:12:31.965 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: down+stopped 2020-04-10T07:12:31.965 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: stopped 2020-04-10T07:12:31.965 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 07:11:14 2020-04-10T07:12:31.966 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:31.966 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror_parent --namespace ls 2020-04-10T07:12:31.966 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:32.008 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test_parent info 2020-04-10T07:12:32.008 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test_parent info 2020-04-10T07:12:32.009 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror_parent --namespace info test_parent 2020-04-10T07:12:32.066 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:rbd image 'test_parent': 2020-04-10T07:12:32.067 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: size 128 MiB in 32 objects 2020-04-10T07:12:32.067 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: order 22 (4 MiB objects) 2020-04-10T07:12:32.067 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: snapshot_count: 2 2020-04-10T07:12:32.068 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: id: 125615c533a 2020-04-10T07:12:32.068 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: block_name_prefix: rbd_data.125615c533a 2020-04-10T07:12:32.068 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: format: 2 2020-04-10T07:12:32.068 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: features: layering, exclusive-lock, journaling 2020-04-10T07:12:32.068 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: op_features: 2020-04-10T07:12:32.069 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: flags: 2020-04-10T07:12:32.069 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: create_timestamp: Fri Apr 10 07:09:42 2020 2020-04-10T07:12:32.069 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: access_timestamp: Fri Apr 10 07:09:42 2020 2020-04-10T07:12:32.069 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: modify_timestamp: Fri Apr 10 07:09:42 2020 2020-04-10T07:12:32.070 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: journal: 125615c533a 2020-04-10T07:12:32.070 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring state: enabled 2020-04-10T07:12:32.070 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring mode: snapshot 2020-04-10T07:12:32.070 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring global id: dd523e97-79dd-49a7-b2c9-df255e9660d4 2020-04-10T07:12:32.070 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring primary: true 2020-04-10T07:12:32.071 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:32.071 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test_parent journal status 2020-04-10T07:12:32.072 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror_parent --namespace journal status --image test_parent 2020-04-10T07:12:32.072 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:32.072 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test_parent journal status 2020-04-10T07:12:32.138 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:minimum_set: 0 2020-04-10T07:12:32.138 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:active_set: 0 2020-04-10T07:12:32.138 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:registered clients: 2020-04-10T07:12:32.138 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: [id=, commit_position=[positions=[[object_number=1, tag_tid=6, entry_tid=29], [object_number=0, tag_tid=6, entry_tid=28], [object_number=3, tag_tid=6, entry_tid=27], [object_number=2, tag_tid=6, entry_tid=26]]], state=connected] 2020-04-10T07:12:32.141 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:32.142 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster2 mirror_parent rbd_mirroring omap vals 2020-04-10T07:12:32.142 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:32.142 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2 mirror_parent rbd_mirroring omap vals 2020-04-10T07:12:32.142 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rados --cluster cluster2 -p mirror_parent --namespace listomapvals rbd_mirroring 2020-04-10T07:12:32.182 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:global_dd523e97-79dd-49a7-b2c9-df255e9660d4 2020-04-10T07:12:32.182 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (15 bytes) : 2020-04-10T07:12:32.182 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 0b 00 00 00 31 32 35 36 31 35 63 35 33 33 61 |....125615c533a| 2020-04-10T07:12:32.183 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000000f 2020-04-10T07:12:32.183 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:32.183 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image_125615c533a 2020-04-10T07:12:32.183 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (48 bytes) : 2020-04-10T07:12:32.184 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 02 01 2a 00 00 00 24 00 00 00 64 64 35 32 33 65 |..*...$...dd523e| 2020-04-10T07:12:32.184 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 39 37 2d 37 39 64 64 2d 34 39 61 37 2d 62 32 63 |97-79dd-49a7-b2c| 2020-04-10T07:12:32.184 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 39 2d 64 66 32 35 35 65 39 36 36 30 64 34 01 01 |9-df255e9660d4..| 2020-04-10T07:12:32.184 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 2020-04-10T07:12:32.184 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:32.184 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:mirror_mode 2020-04-10T07:12:32.185 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (4 bytes) : 2020-04-10T07:12:32.185 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 00 00 00 |....| 2020-04-10T07:12:32.185 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000004 2020-04-10T07:12:32.185 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:32.185 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:mirror_peer_39a2eac0-9f81-4c3b-a329-98e40ae52daf 2020-04-10T07:12:32.185 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (136 bytes) : 2020-04-10T07:12:32.185 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 02 01 82 00 00 00 24 00 00 00 33 39 61 32 65 61 |......$...39a2ea| 2020-04-10T07:12:32.186 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 63 30 2d 39 66 38 31 2d 34 63 33 62 2d 61 33 32 |c0-9f81-4c3b-a32| 2020-04-10T07:12:32.186 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 39 2d 39 38 65 34 30 61 65 35 32 64 61 66 0c 00 |9-98e40ae52daf..| 2020-04-10T07:12:32.186 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 63 6c 75 73 74 65 72 31 2d 44 4e 45 0d 00 |..cluster1-DNE..| 2020-04-10T07:12:32.186 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 00 00 63 6c 69 65 6e 74 2e 6d 69 72 72 6f 72 ff |..client.mirror.| 2020-04-10T07:12:32.186 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 ff ff ff ff ff ff ff 02 24 00 00 00 66 61 35 36 |........$...fa56| 2020-04-10T07:12:32.187 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000060 37 64 39 33 2d 66 34 36 35 2d 34 37 36 34 2d 39 |7d93-f465-4764-9| 2020-04-10T07:12:32.187 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000070 33 63 37 2d 31 32 39 64 39 39 33 63 66 38 61 31 |3c7-129d993cf8a1| 2020-04-10T07:12:32.187 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000080 0d 1c 90 5e 91 a2 26 30 |...^..&0| 2020-04-10T07:12:32.187 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000088 2020-04-10T07:12:32.187 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:32.187 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:mirror_uuid 2020-04-10T07:12:32.188 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (36 bytes) : 2020-04-10T07:12:32.188 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 65 31 35 31 36 36 31 64 2d 39 61 64 63 2d 34 34 |e151661d-9adc-44| 2020-04-10T07:12:32.188 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 34 38 2d 38 35 37 34 2d 38 39 30 64 32 38 34 37 |48-8574-890d2847| 2020-04-10T07:12:32.188 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 63 37 39 30 |c790| 2020-04-10T07:12:32.188 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000024 2020-04-10T07:12:32.188 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:32.189 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:remote_status_global_dd523e97-79dd-49a7-b2c9-df255e9660d4_fa567d93-f465-4764-93c7-129d993cf8a1 2020-04-10T07:12:32.189 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (77 bytes) : 2020-04-10T07:12:32.189 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 2c 11 00 00 00 00 00 00 01 |..,....,........| 2020-04-10T07:12:32.189 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 77 9a 24 e6 10 00 |..........w.$...| 2020-04-10T07:12:32.189 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T07:12:32.189 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 15 00 00 00 06 07 00 00 00 73 74 6f |.............sto| 2020-04-10T07:12:32.189 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 70 70 65 64 12 1c 90 5e 88 9e a9 14 00 |pped...^.....| 2020-04-10T07:12:32.190 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000004d 2020-04-10T07:12:32.190 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:32.190 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:status_global_dd523e97-79dd-49a7-b2c9-df255e9660d4 2020-04-10T07:12:32.190 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (92 bytes) : 2020-04-10T07:12:32.190 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 aa 11 00 00 00 00 00 00 01 |..,.............| 2020-04-10T07:12:32.190 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 dd 09 3b 03 10 00 |............;...| 2020-04-10T07:12:32.191 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T07:12:32.191 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 24 00 00 00 06 16 00 00 00 6c 6f 63 |....$........loc| 2020-04-10T07:12:32.191 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 61 6c 20 69 6d 61 67 65 20 69 73 20 70 72 69 6d |al image is prim| 2020-04-10T07:12:32.191 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 61 72 79 5f 1c 90 5e 44 08 1e 12 00 |ary_..^D....| 2020-04-10T07:12:32.191 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000005c 2020-04-10T07:12:32.191 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:32.192 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:32.192 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster2 mirror_parent ns1 images 2020-04-10T07:12:32.193 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:32.193 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2 mirror_parent ns1 images 2020-04-10T07:12:32.193 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror_parent --namespace ns1 ls -l 2020-04-10T07:12:32.219 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: namespace 'ns1' does not exist. 2020-04-10T07:12:32.219 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: listing images failed: (2) No such file or directory 2020-04-10T07:12:32.222 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:32.222 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster2 mirror_parentns1 mirror pool info 2020-04-10T07:12:32.222 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:32.223 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2 mirror_parentns1 mirror pool info 2020-04-10T07:12:32.223 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror_parent --namespace ns1 mirror pool info 2020-04-10T07:12:32.259 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: namespace 'ns1' does not exist. 2020-04-10T07:12:32.263 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:32.263 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster2 mirror_parentns1 mirror pool status 2020-04-10T07:12:32.263 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:32.263 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2 mirror_parentns1 mirror pool status 2020-04-10T07:12:32.263 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster2 -p mirror_parent --namespace ns1 mirror pool status --verbose 2020-04-10T07:12:32.295 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: namespace 'ns1' does not exist. 2020-04-10T07:12:32.300 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:32.300 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror_parent --namespace ns1 ls 2020-04-10T07:12:32.300 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:32.334 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2 mirror_parent ns1 rbd_mirroring omap vals 2020-04-10T07:12:32.334 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rados --cluster cluster2 -p mirror_parent --namespace ns1 listomapvals rbd_mirroring 2020-04-10T07:12:32.334 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster2 mirror_parent ns1 rbd_mirroring omap vals 2020-04-10T07:12:32.367 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:error getting omap keys mirror_parent/rbd_mirroring: (2) No such file or directory 2020-04-10T07:12:32.370 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:32.371 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster2 mirror_parent ns2 images 2020-04-10T07:12:32.371 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:32.371 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2 mirror_parent ns2 images 2020-04-10T07:12:32.371 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror_parent --namespace ns2 ls -l 2020-04-10T07:12:32.403 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: namespace 'ns2' does not exist. 2020-04-10T07:12:32.403 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: listing images failed: (2) No such file or directory 2020-04-10T07:12:32.406 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:32.406 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster2 mirror_parentns2 mirror pool info 2020-04-10T07:12:32.406 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:32.407 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2 mirror_parentns2 mirror pool info 2020-04-10T07:12:32.407 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror_parent --namespace ns2 mirror pool info 2020-04-10T07:12:32.441 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: namespace 'ns2' does not exist. 2020-04-10T07:12:32.443 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:32.444 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2 mirror_parentns2 mirror pool status 2020-04-10T07:12:32.444 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster2 -p mirror_parent --namespace ns2 mirror pool status --verbose 2020-04-10T07:12:32.444 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:32.444 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster2 mirror_parentns2 mirror pool status 2020-04-10T07:12:32.481 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: namespace 'ns2' does not exist. 2020-04-10T07:12:32.484 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:32.485 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:32.485 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror_parent --namespace ns2 ls 2020-04-10T07:12:32.524 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster2 mirror_parent ns2 rbd_mirroring omap vals 2020-04-10T07:12:32.525 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2 mirror_parent ns2 rbd_mirroring omap vals 2020-04-10T07:12:32.525 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rados --cluster cluster2 -p mirror_parent --namespace ns2 listomapvals rbd_mirroring 2020-04-10T07:12:32.561 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:error getting omap keys mirror_parent/rbd_mirroring: (2) No such file or directory 2020-04-10T07:12:32.565 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:32.565 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local ret 2020-04-10T07:12:32.565 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:12:32.565 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:32.567 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_pid_file cluster1:0 2020-04-10T07:12:32.568 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T07:12:32.568 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:12:32.568 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T07:12:32.568 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T07:12:32.568 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:12:32.568 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:12:32.569 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:12:32.569 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:12:32.569 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T07:12:32.569 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:12:32.569 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:12:32.570 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:12:32.570 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:12:32.570 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 pid file 2020-04-10T07:12:32.587 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.pid 2020-04-10T07:12:32.587 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pid_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.pid 2020-04-10T07:12:32.587 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ ! -e /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.pid ] 2020-04-10T07:12:32.587 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1 rbd-mirror not running or unknown (/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.pid not exist) 2020-04-10T07:12:32.588 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ continue 2020-04-10T07:12:32.588 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_pid_file cluster1:1 2020-04-10T07:12:32.588 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:1 2020-04-10T07:12:32.588 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:12:32.588 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 cluster instance 2020-04-10T07:12:32.588 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T07:12:32.588 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:12:32.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:12:32.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:12:32.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:12:32.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T07:12:32.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:12:32.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:12:32.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:12:32.590 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:12:32.590 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 pid file 2020-04-10T07:12:32.590 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster1 rbd-mirror not running or unknown (/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.pid not exist) 2020-04-10T07:12:32.610 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.pid 2020-04-10T07:12:32.610 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pid_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.pid 2020-04-10T07:12:32.610 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ ! -e /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.pid ] 2020-04-10T07:12:32.611 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1 rbd-mirror not running or unknown (/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.pid not exist) 2020-04-10T07:12:32.611 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ continue 2020-04-10T07:12:32.611 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_pid_file cluster1:2 2020-04-10T07:12:32.611 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:2 2020-04-10T07:12:32.611 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:12:32.611 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 cluster instance 2020-04-10T07:12:32.611 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T07:12:32.612 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:12:32.612 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:12:32.612 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:12:32.612 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:12:32.612 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T07:12:32.613 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:12:32.613 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:12:32.613 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:12:32.613 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:12:32.613 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 pid file 2020-04-10T07:12:32.614 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster1 rbd-mirror not running or unknown (/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.pid not exist) 2020-04-10T07:12:32.631 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.pid 2020-04-10T07:12:32.632 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pid_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.pid 2020-04-10T07:12:32.632 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ ! -e /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.pid ] 2020-04-10T07:12:32.632 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1 rbd-mirror not running or unknown (/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.pid not exist) 2020-04-10T07:12:32.632 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ continue 2020-04-10T07:12:32.632 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_pid_file cluster1:3 2020-04-10T07:12:32.632 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:3 2020-04-10T07:12:32.633 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:12:32.633 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:3 cluster instance 2020-04-10T07:12:32.633 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:3 2020-04-10T07:12:32.633 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:12:32.633 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:12:32.633 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:12:32.634 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:12:32.634 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 3 = cluster1:3 ] 2020-04-10T07:12:32.634 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T07:12:32.634 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T07:12:32.634 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=3 2020-04-10T07:12:32.634 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T07:12:32.634 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.3 pid file 2020-04-10T07:12:32.635 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster1 rbd-mirror not running or unknown (/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.pid not exist) 2020-04-10T07:12:32.650 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.pid 2020-04-10T07:12:32.651 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pid_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.pid 2020-04-10T07:12:32.651 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ ! -e /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.pid ] 2020-04-10T07:12:32.651 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1 rbd-mirror not running or unknown (/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.pid not exist) 2020-04-10T07:12:32.651 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ continue 2020-04-10T07:12:32.651 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T07:12:32.651 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster1 rbd-mirror not running or unknown (/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.pid not exist) 2020-04-10T07:12:32.653 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_pid_file cluster2:0 2020-04-10T07:12:32.653 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2:0 2020-04-10T07:12:32.653 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:12:32.654 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:0 cluster instance 2020-04-10T07:12:32.654 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:0 2020-04-10T07:12:32.654 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:12:32.654 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:12:32.654 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:12:32.654 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:12:32.655 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster2:0 ] 2020-04-10T07:12:32.655 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster2 2020-04-10T07:12:32.655 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:12:32.655 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T07:12:32.655 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T07:12:32.655 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster2 --name client.mirror.0 pid file 2020-04-10T07:12:32.674 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster2-client.mirror.0.pid 2020-04-10T07:12:32.674 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pid_file=/var/run/ceph/rbd-mirror.cluster2-client.mirror.0.pid 2020-04-10T07:12:32.674 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ ! -e /var/run/ceph/rbd-mirror.cluster2-client.mirror.0.pid ] 2020-04-10T07:12:32.675 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2 rbd-mirror not running or unknown (/var/run/ceph/rbd-mirror.cluster2-client.mirror.0.pid not exist) 2020-04-10T07:12:32.675 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ continue 2020-04-10T07:12:32.675 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_pid_file cluster2:1 2020-04-10T07:12:32.675 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2:1 2020-04-10T07:12:32.675 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:12:32.675 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:1 cluster instance 2020-04-10T07:12:32.675 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:1 2020-04-10T07:12:32.675 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:12:32.676 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:12:32.676 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:12:32.676 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:12:32.676 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster2:1 ] 2020-04-10T07:12:32.677 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster2 2020-04-10T07:12:32.677 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:12:32.677 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T07:12:32.677 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T07:12:32.677 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster2 --name client.mirror.1 pid file 2020-04-10T07:12:32.677 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster2 rbd-mirror not running or unknown (/var/run/ceph/rbd-mirror.cluster2-client.mirror.0.pid not exist) 2020-04-10T07:12:32.698 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster2-client.mirror.1.pid 2020-04-10T07:12:32.698 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pid_file=/var/run/ceph/rbd-mirror.cluster2-client.mirror.1.pid 2020-04-10T07:12:32.698 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ ! -e /var/run/ceph/rbd-mirror.cluster2-client.mirror.1.pid ] 2020-04-10T07:12:32.698 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2 rbd-mirror not running or unknown (/var/run/ceph/rbd-mirror.cluster2-client.mirror.1.pid not exist) 2020-04-10T07:12:32.699 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ continue 2020-04-10T07:12:32.699 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_pid_file cluster2:2 2020-04-10T07:12:32.699 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2:2 2020-04-10T07:12:32.699 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:12:32.699 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:2 cluster instance 2020-04-10T07:12:32.699 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:2 2020-04-10T07:12:32.699 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T07:12:32.700 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:12:32.700 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:12:32.700 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:12:32.700 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster2:2 ] 2020-04-10T07:12:32.700 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster2 2020-04-10T07:12:32.700 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:12:32.700 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:12:32.700 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:12:32.701 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster2 --name client.mirror.2 pid file 2020-04-10T07:12:32.701 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster2 rbd-mirror not running or unknown (/var/run/ceph/rbd-mirror.cluster2-client.mirror.1.pid not exist) 2020-04-10T07:12:32.717 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster2-client.mirror.2.pid 2020-04-10T07:12:32.717 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pid_file=/var/run/ceph/rbd-mirror.cluster2-client.mirror.2.pid 2020-04-10T07:12:32.717 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ ! -e /var/run/ceph/rbd-mirror.cluster2-client.mirror.2.pid ] 2020-04-10T07:12:32.717 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pid 2020-04-10T07:12:32.717 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cat /var/run/ceph/rbd-mirror.cluster2-client.mirror.2.pid 2020-04-10T07:12:32.718 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ pid=19197 2020-04-10T07:12:32.718 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -z 19197 ] 2020-04-10T07:12:32.719 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo rbd-mirror process in ps output: 2020-04-10T07:12:32.719 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ps auxww 2020-04-10T07:12:32.719 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ awk -v pid=19197 NR == 1 {print} $2 == pid {print; exit 1} 2020-04-10T07:12:32.719 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: rbd-mirror process in ps output: 2020-04-10T07:12:32.739 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND 2020-04-10T07:12:32.739 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:ubuntu 19197 0.2 0.1 803940 43312 ? Ssl 07:08 0:00 rbd-mirror --cluster cluster2 --id mirror.2 --rbd-mirror-delete-retry-interval=5 --rbd-mirror-image-state-check-interval=5 --rbd-mirror-journal-poll-age=1 --rbd-mirror-pool-replayers-refresh-interval=5 --debug-rbd=30 --debug-journaler=30 --debug-rbd_mirror=30 --daemonize=true 2020-04-10T07:12:32.739 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T07:12:32.739 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T07:12:32.739 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster2:2 cluster2 2020-04-10T07:12:32.740 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster2:2 2020-04-10T07:12:32.740 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T07:12:32.745 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T07:12:32.745 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:2 local_cluster instance 2020-04-10T07:12:32.745 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:2 2020-04-10T07:12:32.745 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T07:12:32.746 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T07:12:32.746 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T07:12:32.746 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:12:32.746 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster2:2 ] 2020-04-10T07:12:32.746 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster2 2020-04-10T07:12:32.746 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster2 2020-04-10T07:12:32.746 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T07:12:32.746 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T07:12:32.747 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster2 --name client.mirror.2 admin socket 2020-04-10T07:12:32.759 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster2-client.mirror.2.asok 2020-04-10T07:12:32.759 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster2-client.mirror.2.asok 2020-04-10T07:12:32.760 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ ! -S /var/run/ceph/rbd-mirror.cluster2-client.mirror.2.asok ] 2020-04-10T07:12:32.760 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2 rbd-mirror status 2020-04-10T07:12:32.760 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster2-client.mirror.2.asok rbd mirror status 2020-04-10T07:12:32.760 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster2 rbd-mirror status 2020-04-10T07:15:46.197 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T07:15:46.194+0000 7fefd66bd700 -1 reset not still connected to 0x556b42443860 2020-04-10T07:15:46.198 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T07:15:46.194+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283aa90 2020-04-10T07:15:46.199 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T07:15:46.194+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283b930 2020-04-10T07:15:46.199 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T07:15:46.194+0000 7fefd66bd700 -1 reset not still connected to 0x556b4313a820 2020-04-10T07:55:16.259 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T07:55:16.250+0000 7fefd66bd700 -1 reset not still connected to 0x556b42443860 2020-04-10T07:55:16.285 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T07:55:16.254+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283aa90 2020-04-10T07:55:16.285 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T07:55:16.254+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283b930 2020-04-10T07:55:16.286 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T07:55:16.254+0000 7fefd66bd700 -1 reset not still connected to 0x556b4313a820 2020-04-10T07:56:26.257 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T07:56:26.249+0000 7fefd66bd700 -1 reset not still connected to 0x556b4313a820 2020-04-10T07:56:26.264 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T07:56:26.257+0000 7fefd66bd700 -1 reset not still connected to 0x556b42443860 2020-04-10T07:56:26.267 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T07:56:26.261+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283aa90 2020-04-10T07:56:26.267 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T07:56:26.261+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283b930 2020-04-10T07:56:26.267 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T07:56:26.261+0000 7fefd66bd700 -1 reset not still connected to 0x556b4313a820 2020-04-10T08:05:56.291 INFO:tasks.ceph.cluster1.osd.2.smithi101.stderr:2020-04-10T08:05:56.283+0000 7f8105502700 -1 reset not still connected to 0x55d7e67c2000 2020-04-10T08:20:21.307 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T08:20:21.300+0000 7fefd66bd700 -1 reset not still connected to 0x556b42443860 2020-04-10T08:20:21.335 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T08:20:21.300+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283aa90 2020-04-10T08:20:21.335 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T08:20:21.300+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283b930 2020-04-10T08:20:21.335 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T08:20:21.300+0000 7fefd66bd700 -1 reset not still connected to 0x556b4313a820 2020-04-10T08:21:41.425 INFO:tasks.ceph.cluster2.osd.1.smithi204.stderr:2020-04-10T08:21:41.418+0000 7f5dcede3700 -1 reset not still connected to 0x559a9f61e080 2020-04-10T08:27:31.315 INFO:tasks.ceph.cluster1.osd.2.smithi101.stderr:2020-04-10T08:27:31.308+0000 7f8105502700 -1 reset not still connected to 0x55d7e67c2000 2020-04-10T08:33:35.166 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T08:33:35.156+0000 7fefd66bd700 -1 reset not still connected to 0x556b3e67fe10 2020-04-10T08:33:35.188 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T08:33:35.156+0000 7fefd66bd700 -1 reset not still connected to 0x556b42443860 2020-04-10T08:33:35.188 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T08:33:35.156+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283aa90 2020-04-10T08:33:35.189 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T08:33:35.156+0000 7fefd66bd700 -1 reset not still connected to 0x556b4313a820 2020-04-10T08:41:11.350 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T08:41:11.345+0000 7fefd66bd700 -1 reset not still connected to 0x556b42443860 2020-04-10T08:41:11.352 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T08:41:11.345+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283aa90 2020-04-10T08:41:11.352 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T08:41:11.345+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283b930 2020-04-10T08:41:11.352 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T08:41:11.345+0000 7fefd66bd700 -1 reset not still connected to 0x556b4313a820 2020-04-10T08:48:11.362 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T08:48:11.355+0000 7fefd66bd700 -1 reset not still connected to 0x556b42443860 2020-04-10T08:48:11.362 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T08:48:11.355+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283aa90 2020-04-10T08:48:11.362 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T08:48:11.355+0000 7fefd66bd700 -1 reset not still connected to 0x556b4313a820 2020-04-10T08:53:26.368 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T08:53:26.362+0000 7fefd66bd700 -1 reset not still connected to 0x556b42443860 2020-04-10T08:53:26.370 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T08:53:26.366+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283aa90 2020-04-10T08:53:26.370 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T08:53:26.366+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283b930 2020-04-10T08:53:26.370 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T08:53:26.366+0000 7fefd66bd700 -1 reset not still connected to 0x556b4313a820 2020-04-10T08:59:06.376 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T08:59:06.369+0000 7fefd66bd700 -1 reset not still connected to 0x556b42443860 2020-04-10T08:59:06.395 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T08:59:06.369+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283aa90 2020-04-10T08:59:06.395 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T08:59:06.369+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283b930 2020-04-10T08:59:06.395 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T08:59:06.369+0000 7fefd66bd700 -1 reset not still connected to 0x556b4313a820 2020-04-10T09:17:26.407 INFO:tasks.ceph.cluster1.osd.2.smithi101.stderr:2020-04-10T09:17:26.402+0000 7f8105502700 -1 reset not still connected to 0x55d7e67c2000 2020-04-10T09:25:36.419 INFO:tasks.ceph.cluster1.osd.2.smithi101.stderr:2020-04-10T09:25:36.414+0000 7f8105502700 -1 reset not still connected to 0x55d7e67c2000 2020-04-10T09:30:21.430 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T09:30:21.416+0000 7fefd66bd700 -1 reset not still connected to 0x556b42443860 2020-04-10T09:30:21.430 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T09:30:21.416+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283aa90 2020-04-10T09:30:21.430 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T09:30:21.420+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283b930 2020-04-10T09:30:21.430 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T09:30:21.420+0000 7fefd66bd700 -1 reset not still connected to 0x556b4313a820 2020-04-10T09:37:00.431 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T09:37:00.426+0000 7fefd66bd700 -1 reset not still connected to 0x556b42444700 2020-04-10T09:37:00.433 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T09:37:00.430+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283bd40 2020-04-10T09:38:26.439 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T09:38:26.433+0000 7fefd66bd700 -1 reset not still connected to 0x556b42443860 2020-04-10T09:38:26.439 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T09:38:26.433+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283aa90 2020-04-10T09:38:26.440 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T09:38:26.433+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283b930 2020-04-10T09:38:26.440 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T09:38:26.433+0000 7fefd66bd700 -1 reset not still connected to 0x556b4313a820 2020-04-10T09:47:56.445 INFO:tasks.ceph.cluster1.osd.2.smithi101.stderr:2020-04-10T09:47:56.439+0000 7f8105502700 -1 reset not still connected to 0x55d7e67c2000 2020-04-10T09:56:10.387 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T09:56:10.365+0000 7fefd66bd700 -1 reset not still connected to 0x556b42444700 2020-04-10T09:56:10.403 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T09:56:10.365+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283bd40 2020-04-10T09:58:56.475 INFO:tasks.ceph.cluster1.osd.2.smithi101.stderr:2020-04-10T09:58:56.467+0000 7f8105502700 -1 reset not still connected to 0x55d7e67c2000 2020-04-10T10:05:31.487 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T10:05:31.480+0000 7fefd66bd700 -1 reset not still connected to 0x556b42444700 2020-04-10T10:05:31.488 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T10:05:31.480+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283bd40 2020-04-10T10:06:01.478 INFO:tasks.ceph.cluster1.osd.2.smithi101.stderr:2020-04-10T10:06:01.469+0000 7f8105502700 -1 reset not still connected to 0x55d7e67c2000 2020-04-10T10:06:08.366 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:Terminated 2020-04-10T10:06:08.366 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cleanup 143 2020-04-10T10:06:08.367 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local error_code=143 2020-04-10T10:06:08.367 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set +e 2020-04-10T10:06:08.367 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 143 -ne 0 ] 2020-04-10T10:06:08.367 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ status 2020-04-10T10:06:08.367 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster daemon image_pool image_ns image 2020-04-10T10:06:08.367 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1 status 2020-04-10T10:06:08.367 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= ceph --cluster cluster1 -s 2020-04-10T10:06:08.368 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster1 status 2020-04-10T10:06:08.772 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: cluster: 2020-04-10T10:06:08.772 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: id: 5d7eff56-55fb-4da2-8b7d-3ec54a26cfc2 2020-04-10T10:06:08.772 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: health: HEALTH_OK 2020-04-10T10:06:08.772 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:08.772 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: services: 2020-04-10T10:06:08.772 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mon: 1 daemons, quorum a (age 3h) 2020-04-10T10:06:08.773 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mgr: x(active, since 3h) 2020-04-10T10:06:08.773 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: osd: 3 osds: 3 up (since 3h), 3 in (since 3h) 2020-04-10T10:06:08.773 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:08.773 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: data: 2020-04-10T10:06:08.773 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: pools: 4 pools, 137 pgs 2020-04-10T10:06:08.773 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: objects: 524 objects, 720 MiB 2020-04-10T10:06:08.773 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: usage: 3.0 GiB used, 267 GiB / 270 GiB avail 2020-04-10T10:06:08.774 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: pgs: 137 active+clean 2020-04-10T10:06:08.774 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:08.774 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: io: 2020-04-10T10:06:08.774 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: client: 8.2 KiB/s rd, 1023 B/s wr, 8 op/s rd, 0 op/s wr 2020-04-10T10:06:08.774 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:08.782 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= ceph --cluster cluster1 service dump 2020-04-10T10:06:09.095 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:{ 2020-04-10T10:06:09.095 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "epoch": 8, 2020-04-10T10:06:09.095 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "modified": "2020-04-10T07:11:18.743912+0000", 2020-04-10T10:06:09.095 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "services": {} 2020-04-10T10:06:09.096 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:} 2020-04-10T10:06:09.106 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= ceph --cluster cluster1 service status 2020-04-10T10:06:09.418 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:{} 2020-04-10T10:06:09.428 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:09.428 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster1 mirror images 2020-04-10T10:06:09.429 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:09.429 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1 mirror images 2020-04-10T10:06:09.429 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror --namespace ls -l 2020-04-10T10:06:09.526 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:NAME SIZE PARENT FMT PROT LOCK 2020-04-10T10:06:09.526 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test 128 MiB 2 2020-04-10T10:06:09.526 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test1 128 MiB 2 2020-04-10T10:06:09.526 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test2 128 MiB 2 2020-04-10T10:06:09.526 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test2@snap1 128 MiB 2 2020-04-10T10:06:09.527 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test2@snap2 128 MiB 2 2020-04-10T10:06:09.527 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test2@snap_rename_20 128 MiB 2 2020-04-10T10:06:09.527 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test_clone 128 MiB mirror_parent/test_parent@snap 2 2020-04-10T10:06:09.527 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test_clone1 128 MiB mirror_parent/test_parent@snap 2 2020-04-10T10:06:09.527 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test_clone_v1 128 MiB mirror_parent/test_parent@snap 2 2020-04-10T10:06:09.527 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test_data_pool 128 MiB 2 2020-04-10T10:06:09.527 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test_data_pool@snap1 128 MiB 2 2020-04-10T10:06:09.528 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test_data_pool@snap2 128 MiB 2 2020-04-10T10:06:09.528 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test_force_promote 128 MiB 2 2020-04-10T10:06:09.533 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:09.533 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1 mirror mirror pool info 2020-04-10T10:06:09.533 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror --namespace mirror pool info 2020-04-10T10:06:09.534 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:09.534 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster1 mirror mirror pool info 2020-04-10T10:06:09.565 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Mode: image 2020-04-10T10:06:09.566 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Site Name: cluster1-DNE 2020-04-10T10:06:09.566 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:09.566 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Peer Sites: 2020-04-10T10:06:09.566 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:09.566 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:UUID: 2571f49c-d767-4112-ae12-d8b71263d58b 2020-04-10T10:06:09.566 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Name: cluster2-DNE 2020-04-10T10:06:09.567 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Mirror UUID: 8253fde2-86f8-443a-b4bf-ec7dae2fafa3 2020-04-10T10:06:09.567 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Direction: rx-tx 2020-04-10T10:06:09.567 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Client: client.mirror 2020-04-10T10:06:09.568 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:09.568 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1 mirror mirror pool status 2020-04-10T10:06:09.568 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster1 -p mirror --namespace mirror pool status --verbose 2020-04-10T10:06:09.568 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:09.568 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster1 mirror mirror pool status 2020-04-10T10:06:09.608 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:health: UNKNOWN 2020-04-10T10:06:09.608 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:daemon health: UNKNOWN 2020-04-10T10:06:09.609 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image health: OK 2020-04-10T10:06:09.609 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:images: 8 total 2020-04-10T10:06:09.609 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 1 replaying 2020-04-10T10:06:09.609 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 7 stopped 2020-04-10T10:06:09.609 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:09.610 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:DAEMONS 2020-04-10T10:06:09.610 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: none 2020-04-10T10:06:09.610 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:09.669 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:IMAGES 2020-04-10T10:06:09.670 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test: 2020-04-10T10:06:09.670 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: global_id: a9af7d9e-1404-4730-9e18-c0418a915346 2020-04-10T10:06:09.670 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: down+stopped 2020-04-10T10:06:09.670 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: stopped 2020-04-10T10:06:09.671 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 07:11:16 2020-04-10T10:06:09.671 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: peer_sites: 2020-04-10T10:06:09.671 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: name: cluster2-DNE 2020-04-10T10:06:09.671 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+stopped 2020-04-10T10:06:09.672 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: local image is primary 2020-04-10T10:06:09.672 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 10:06:06 2020-04-10T10:06:09.672 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:09.672 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test1: 2020-04-10T10:06:09.672 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: global_id: 56419286-13e9-49f6-a20d-2abe6ede083e 2020-04-10T10:06:09.673 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: down+stopped 2020-04-10T10:06:09.673 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: stopped 2020-04-10T10:06:09.673 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 07:11:14 2020-04-10T10:06:09.673 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: peer_sites: 2020-04-10T10:06:09.673 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: name: cluster2-DNE 2020-04-10T10:06:09.674 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+stopped 2020-04-10T10:06:09.674 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: local image is primary 2020-04-10T10:06:09.674 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 10:06:06 2020-04-10T10:06:09.674 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:09.674 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test2: 2020-04-10T10:06:09.675 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: global_id: b583e294-8b69-4e57-9966-783595c454d6 2020-04-10T10:06:09.675 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: down+stopped 2020-04-10T10:06:09.675 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: stopped 2020-04-10T10:06:09.675 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 07:11:15 2020-04-10T10:06:09.675 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: peer_sites: 2020-04-10T10:06:09.676 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: name: cluster2-DNE 2020-04-10T10:06:09.676 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+stopped 2020-04-10T10:06:09.676 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: local image is primary 2020-04-10T10:06:09.676 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 07:12:31 2020-04-10T10:06:09.687 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:09.687 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test_clone: 2020-04-10T10:06:09.687 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: global_id: 003078e2-ed6c-4f36-8bc5-ce0c44ef25d5 2020-04-10T10:06:09.687 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: down+stopped 2020-04-10T10:06:09.688 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: stopped 2020-04-10T10:06:09.688 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 07:11:14 2020-04-10T10:06:09.688 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: peer_sites: 2020-04-10T10:06:09.688 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: name: cluster2-DNE 2020-04-10T10:06:09.689 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+stopped 2020-04-10T10:06:09.689 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: local image is primary 2020-04-10T10:06:09.689 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 10:06:06 2020-04-10T10:06:09.689 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:09.689 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test_clone1: 2020-04-10T10:06:09.690 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: global_id: 4b1961cb-5406-443f-90f1-3627ac4ffffd 2020-04-10T10:06:09.690 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: down+stopped 2020-04-10T10:06:09.693 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: local image is primary 2020-04-10T10:06:09.693 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 07:11:15 2020-04-10T10:06:09.693 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: peer_sites: 2020-04-10T10:06:09.693 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: name: cluster2-DNE 2020-04-10T10:06:09.693 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+replaying 2020-04-10T10:06:09.693 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: replaying, {"replay_state": "idle", "remote_snapshot_timestamp": 1586502598, "local_snapshot_timestamp": 1586502598} 2020-04-10T10:06:09.694 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 07:12:36 2020-04-10T10:06:09.695 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:09.695 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test_clone_v1: 2020-04-10T10:06:09.695 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: global_id: 70d86755-f73a-4bbe-9493-1b79a191cc0c 2020-04-10T10:06:09.695 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: down+stopped 2020-04-10T10:06:09.695 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: stopped 2020-04-10T10:06:09.696 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 07:11:16 2020-04-10T10:06:09.696 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: peer_sites: 2020-04-10T10:06:09.696 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: name: cluster2-DNE 2020-04-10T10:06:09.696 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+stopped 2020-04-10T10:06:09.696 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: local image is primary 2020-04-10T10:06:09.696 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 07:12:31 2020-04-10T10:06:09.696 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:09.697 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test_data_pool: 2020-04-10T10:06:09.697 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: global_id: a8183526-4331-4f0d-af4a-7c93c501e927 2020-04-10T10:06:09.697 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: down+stopped 2020-04-10T10:06:09.697 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: stopped 2020-04-10T10:06:09.697 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 07:11:14 2020-04-10T10:06:09.697 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: peer_sites: 2020-04-10T10:06:09.697 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: name: cluster2-DNE 2020-04-10T10:06:09.698 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+stopped 2020-04-10T10:06:09.698 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: local image is primary 2020-04-10T10:06:09.698 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 10:06:06 2020-04-10T10:06:09.698 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:09.698 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test_force_promote: 2020-04-10T10:06:09.698 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: global_id: b192e243-6abc-405a-b798-b60b9d74691e 2020-04-10T10:06:09.698 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: down+stopped 2020-04-10T10:06:09.699 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: local image is primary 2020-04-10T10:06:09.699 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 07:11:16 2020-04-10T10:06:09.699 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: peer_sites: 2020-04-10T10:06:09.699 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: name: cluster2-DNE 2020-04-10T10:06:09.699 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+stopped 2020-04-10T10:06:09.699 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: local image is primary 2020-04-10T10:06:09.699 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 07:12:31 2020-04-10T10:06:09.699 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:09.700 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:09.700 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror --namespace ls 2020-04-10T10:06:09.735 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test info 2020-04-10T10:06:09.735 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test info 2020-04-10T10:06:09.735 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror --namespace info test 2020-04-10T10:06:09.813 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:rbd image 'test': 2020-04-10T10:06:09.814 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: size 128 MiB in 32 objects 2020-04-10T10:06:09.814 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: order 22 (4 MiB objects) 2020-04-10T10:06:09.814 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: snapshot_count: 14 2020-04-10T10:06:09.814 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: id: 114bcee179da 2020-04-10T10:06:09.814 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: block_name_prefix: rbd_data.114bcee179da 2020-04-10T10:06:09.814 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: format: 2 2020-04-10T10:06:09.815 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: features: layering, exclusive-lock, journaling, non-primary 2020-04-10T10:06:09.815 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: op_features: 2020-04-10T10:06:09.815 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: flags: 2020-04-10T10:06:09.815 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: create_timestamp: Fri Apr 10 07:08:05 2020 2020-04-10T10:06:09.815 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: access_timestamp: Fri Apr 10 07:09:20 2020 2020-04-10T10:06:09.815 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: modify_timestamp: Fri Apr 10 07:08:05 2020 2020-04-10T10:06:09.815 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: journal: 114bcee179da 2020-04-10T10:06:09.816 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring state: enabled 2020-04-10T10:06:09.816 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring mode: snapshot 2020-04-10T10:06:09.816 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring global id: a9af7d9e-1404-4730-9e18-c0418a915346 2020-04-10T10:06:09.816 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring primary: false 2020-04-10T10:06:09.816 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:09.816 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test journal status 2020-04-10T10:06:09.817 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:09.817 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test journal status 2020-04-10T10:06:09.817 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror --namespace journal status --image test 2020-04-10T10:06:09.894 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:minimum_set: 0 2020-04-10T10:06:09.894 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:active_set: 0 2020-04-10T10:06:09.895 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:registered clients: 2020-04-10T10:06:09.895 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: [id=, commit_position=[positions=[[object_number=1, tag_tid=7, entry_tid=1], [object_number=0, tag_tid=7, entry_tid=0], [object_number=3, tag_tid=4, entry_tid=99], [object_number=2, tag_tid=4, entry_tid=98]]], state=connected] 2020-04-10T10:06:09.899 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:09.899 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test1 info 2020-04-10T10:06:09.899 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:09.899 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test1 info 2020-04-10T10:06:09.899 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror --namespace info test1 2020-04-10T10:06:09.962 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:rbd image 'test1': 2020-04-10T10:06:09.962 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: size 128 MiB in 32 objects 2020-04-10T10:06:09.962 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: order 22 (4 MiB objects) 2020-04-10T10:06:09.962 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: snapshot_count: 2 2020-04-10T10:06:09.962 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: id: 113f5aed03a0 2020-04-10T10:06:09.962 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: block_name_prefix: rbd_data.113f5aed03a0 2020-04-10T10:06:09.963 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: format: 2 2020-04-10T10:06:09.963 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: features: layering, exclusive-lock, journaling, non-primary 2020-04-10T10:06:09.963 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: op_features: 2020-04-10T10:06:09.963 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: flags: 2020-04-10T10:06:09.964 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: create_timestamp: Fri Apr 10 07:06:41 2020 2020-04-10T10:06:09.964 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: access_timestamp: Fri Apr 10 07:06:41 2020 2020-04-10T10:06:09.964 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: modify_timestamp: Fri Apr 10 07:06:41 2020 2020-04-10T10:06:09.964 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: journal: 113f5aed03a0 2020-04-10T10:06:09.965 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring state: enabled 2020-04-10T10:06:09.965 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring mode: snapshot 2020-04-10T10:06:09.965 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring global id: 56419286-13e9-49f6-a20d-2abe6ede083e 2020-04-10T10:06:09.965 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring primary: false 2020-04-10T10:06:09.967 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:09.967 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test1 journal status 2020-04-10T10:06:09.967 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror --namespace journal status --image test1 2020-04-10T10:06:09.968 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:09.968 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test1 journal status 2020-04-10T10:06:10.046 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:minimum_set: 0 2020-04-10T10:06:10.046 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:active_set: 0 2020-04-10T10:06:10.047 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:registered clients: 2020-04-10T10:06:10.047 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: [id=, commit_position=[positions=[]], state=connected] 2020-04-10T10:06:10.050 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:10.051 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test2 info 2020-04-10T10:06:10.051 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:10.051 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test2 info 2020-04-10T10:06:10.051 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror --namespace info test2 2020-04-10T10:06:10.116 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:rbd image 'test2': 2020-04-10T10:06:10.116 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: size 128 MiB in 32 objects 2020-04-10T10:06:10.116 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: order 22 (4 MiB objects) 2020-04-10T10:06:10.116 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: snapshot_count: 5 2020-04-10T10:06:10.116 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: id: 11439010b41 2020-04-10T10:06:10.117 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: block_name_prefix: rbd_data.11439010b41 2020-04-10T10:06:10.117 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: format: 2 2020-04-10T10:06:10.117 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: features: layering, exclusive-lock, journaling, non-primary 2020-04-10T10:06:10.117 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: op_features: 2020-04-10T10:06:10.117 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: flags: 2020-04-10T10:06:10.117 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: create_timestamp: Fri Apr 10 07:11:09 2020 2020-04-10T10:06:10.118 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: access_timestamp: Fri Apr 10 07:11:09 2020 2020-04-10T10:06:10.118 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: modify_timestamp: Fri Apr 10 07:11:09 2020 2020-04-10T10:06:10.118 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: journal: 11439010b41 2020-04-10T10:06:10.118 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring state: enabled 2020-04-10T10:06:10.118 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring mode: snapshot 2020-04-10T10:06:10.118 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring global id: b583e294-8b69-4e57-9966-783595c454d6 2020-04-10T10:06:10.118 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring primary: false 2020-04-10T10:06:10.121 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:10.121 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test2 journal status 2020-04-10T10:06:10.121 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:10.121 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test2 journal status 2020-04-10T10:06:10.121 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror --namespace journal status --image test2 2020-04-10T10:06:10.196 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:minimum_set: 0 2020-04-10T10:06:10.196 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:active_set: 0 2020-04-10T10:06:10.197 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:registered clients: 2020-04-10T10:06:10.197 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: [id=, commit_position=[positions=[]], state=connected] 2020-04-10T10:06:10.200 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:10.200 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test_clone info 2020-04-10T10:06:10.200 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:10.201 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test_clone info 2020-04-10T10:06:10.201 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror --namespace info test_clone 2020-04-10T10:06:10.489 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:rbd image 'test_clone': 2020-04-10T10:06:10.490 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: size 128 MiB in 32 objects 2020-04-10T10:06:10.490 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: order 22 (4 MiB objects) 2020-04-10T10:06:10.490 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: snapshot_count: 2 2020-04-10T10:06:10.490 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: id: 113f209cc429 2020-04-10T10:06:10.490 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: block_name_prefix: rbd_data.113f209cc429 2020-04-10T10:06:10.491 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: format: 2 2020-04-10T10:06:10.491 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: features: layering, exclusive-lock, journaling, non-primary 2020-04-10T10:06:10.491 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: op_features: 2020-04-10T10:06:10.491 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: flags: 2020-04-10T10:06:10.491 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: create_timestamp: Fri Apr 10 07:09:49 2020 2020-04-10T10:06:10.491 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: access_timestamp: Fri Apr 10 07:09:49 2020 2020-04-10T10:06:10.491 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: modify_timestamp: Fri Apr 10 07:09:49 2020 2020-04-10T10:06:10.492 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: parent: mirror_parent/test_parent@snap 2020-04-10T10:06:10.492 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: overlap: 128 MiB 2020-04-10T10:06:10.492 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: journal: 113f209cc429 2020-04-10T10:06:10.492 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring state: enabled 2020-04-10T10:06:10.493 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring mode: snapshot 2020-04-10T10:06:10.493 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring global id: 003078e2-ed6c-4f36-8bc5-ce0c44ef25d5 2020-04-10T10:06:10.494 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring primary: false 2020-04-10T10:06:10.497 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:10.497 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test_clone journal status 2020-04-10T10:06:10.497 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror --namespace journal status --image test_clone 2020-04-10T10:06:10.498 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:10.498 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test_clone journal status 2020-04-10T10:06:10.592 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:minimum_set: 0 2020-04-10T10:06:10.592 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:active_set: 0 2020-04-10T10:06:10.593 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:registered clients: 2020-04-10T10:06:10.593 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: [id=, commit_position=[positions=[]], state=connected] 2020-04-10T10:06:10.597 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:10.597 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test_clone1 info 2020-04-10T10:06:10.598 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:10.598 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test_clone1 info 2020-04-10T10:06:10.598 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror --namespace info test_clone1 2020-04-10T10:06:10.684 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:rbd image 'test_clone1': 2020-04-10T10:06:10.684 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: size 128 MiB in 32 objects 2020-04-10T10:06:10.685 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: order 22 (4 MiB objects) 2020-04-10T10:06:10.685 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: snapshot_count: 1 2020-04-10T10:06:10.685 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: id: 134fd0e99700 2020-04-10T10:06:10.685 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: block_name_prefix: rbd_data.134fd0e99700 2020-04-10T10:06:10.685 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: format: 2 2020-04-10T10:06:10.685 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: features: layering, exclusive-lock, journaling 2020-04-10T10:06:10.686 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: op_features: 2020-04-10T10:06:10.686 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: flags: 2020-04-10T10:06:10.686 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: create_timestamp: Fri Apr 10 07:09:57 2020 2020-04-10T10:06:10.686 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: access_timestamp: Fri Apr 10 07:09:57 2020 2020-04-10T10:06:10.686 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: modify_timestamp: Fri Apr 10 07:09:57 2020 2020-04-10T10:06:10.687 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: parent: mirror_parent/test_parent@snap 2020-04-10T10:06:10.687 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: overlap: 128 MiB 2020-04-10T10:06:10.687 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: journal: 134fd0e99700 2020-04-10T10:06:10.687 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring state: enabled 2020-04-10T10:06:10.687 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring mode: snapshot 2020-04-10T10:06:10.688 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring global id: 4b1961cb-5406-443f-90f1-3627ac4ffffd 2020-04-10T10:06:10.688 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring primary: true 2020-04-10T10:06:10.691 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:10.691 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test_clone1 journal status 2020-04-10T10:06:10.691 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror --namespace journal status --image test_clone1 2020-04-10T10:06:10.692 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:10.692 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test_clone1 journal status 2020-04-10T10:06:10.788 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:minimum_set: 0 2020-04-10T10:06:10.788 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:active_set: 0 2020-04-10T10:06:10.788 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:registered clients: 2020-04-10T10:06:10.788 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: [id=, commit_position=[positions=[[object_number=1, tag_tid=1, entry_tid=1], [object_number=0, tag_tid=1, entry_tid=0]]], state=connected] 2020-04-10T10:06:10.792 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:10.792 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test_clone_v1 info 2020-04-10T10:06:10.792 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror --namespace info test_clone_v1 2020-04-10T10:06:10.792 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:10.792 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test_clone_v1 info 2020-04-10T10:06:10.873 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:rbd image 'test_clone_v1': 2020-04-10T10:06:10.873 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: size 128 MiB in 32 objects 2020-04-10T10:06:10.873 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: order 22 (4 MiB objects) 2020-04-10T10:06:10.873 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: snapshot_count: 1 2020-04-10T10:06:10.873 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: id: 114bb4bac5c 2020-04-10T10:06:10.873 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: block_name_prefix: rbd_data.114bb4bac5c 2020-04-10T10:06:10.874 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: format: 2 2020-04-10T10:06:10.874 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: features: layering, exclusive-lock, journaling, non-primary 2020-04-10T10:06:10.874 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: op_features: 2020-04-10T10:06:10.874 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: flags: 2020-04-10T10:06:10.874 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: create_timestamp: Fri Apr 10 07:10:00 2020 2020-04-10T10:06:10.874 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: access_timestamp: Fri Apr 10 07:10:00 2020 2020-04-10T10:06:10.874 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: modify_timestamp: Fri Apr 10 07:10:00 2020 2020-04-10T10:06:10.875 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: parent: mirror_parent/test_parent@snap 2020-04-10T10:06:10.875 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: overlap: 128 MiB 2020-04-10T10:06:10.875 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: journal: 114bb4bac5c 2020-04-10T10:06:10.875 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring state: enabled 2020-04-10T10:06:10.875 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring mode: snapshot 2020-04-10T10:06:10.875 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring global id: 70d86755-f73a-4bbe-9493-1b79a191cc0c 2020-04-10T10:06:10.876 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring primary: false 2020-04-10T10:06:10.880 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:10.880 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test_clone_v1 journal status 2020-04-10T10:06:10.881 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror --namespace journal status --image test_clone_v1 2020-04-10T10:06:10.881 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:10.881 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test_clone_v1 journal status 2020-04-10T10:06:10.979 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:minimum_set: 0 2020-04-10T10:06:10.980 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:active_set: 0 2020-04-10T10:06:10.980 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:registered clients: 2020-04-10T10:06:10.980 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: [id=, commit_position=[positions=[]], state=connected] 2020-04-10T10:06:10.983 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:10.983 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test_data_pool info 2020-04-10T10:06:10.983 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:10.984 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test_data_pool info 2020-04-10T10:06:10.984 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror --namespace info test_data_pool 2020-04-10T10:06:11.048 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:rbd image 'test_data_pool': 2020-04-10T10:06:11.048 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: size 128 MiB in 32 objects 2020-04-10T10:06:11.049 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: order 22 (4 MiB objects) 2020-04-10T10:06:11.049 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: snapshot_count: 4 2020-04-10T10:06:11.049 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: id: 113fde7aeaa 2020-04-10T10:06:11.049 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: data_pool: mirror_parent 2020-04-10T10:06:11.049 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: block_name_prefix: rbd_data.3.113fde7aeaa 2020-04-10T10:06:11.049 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: format: 2 2020-04-10T10:06:11.049 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: features: layering, exclusive-lock, journaling, data-pool, non-primary 2020-04-10T10:06:11.050 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: op_features: 2020-04-10T10:06:11.050 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: flags: 2020-04-10T10:06:11.050 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: create_timestamp: Fri Apr 10 07:10:23 2020 2020-04-10T10:06:11.050 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: access_timestamp: Fri Apr 10 07:10:23 2020 2020-04-10T10:06:11.050 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: modify_timestamp: Fri Apr 10 07:10:23 2020 2020-04-10T10:06:11.050 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: journal: 113fde7aeaa 2020-04-10T10:06:11.051 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring state: enabled 2020-04-10T10:06:11.051 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring mode: snapshot 2020-04-10T10:06:11.051 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring global id: a8183526-4331-4f0d-af4a-7c93c501e927 2020-04-10T10:06:11.051 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring primary: false 2020-04-10T10:06:11.053 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.053 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test_data_pool journal status 2020-04-10T10:06:11.054 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:11.054 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test_data_pool journal status 2020-04-10T10:06:11.054 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror --namespace journal status --image test_data_pool 2020-04-10T10:06:11.124 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:minimum_set: 0 2020-04-10T10:06:11.125 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:active_set: 0 2020-04-10T10:06:11.125 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:registered clients: 2020-04-10T10:06:11.125 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: [id=, commit_position=[positions=[]], state=connected] 2020-04-10T10:06:11.129 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.129 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test_force_promote info 2020-04-10T10:06:11.129 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:11.129 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test_force_promote info 2020-04-10T10:06:11.130 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror --namespace info test_force_promote 2020-04-10T10:06:11.193 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:rbd image 'test_force_promote': 2020-04-10T10:06:11.193 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: size 128 MiB in 32 objects 2020-04-10T10:06:11.194 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: order 22 (4 MiB objects) 2020-04-10T10:06:11.194 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: snapshot_count: 4 2020-04-10T10:06:11.194 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: id: 115186996acb 2020-04-10T10:06:11.194 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: block_name_prefix: rbd_data.115186996acb 2020-04-10T10:06:11.194 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: format: 2 2020-04-10T10:06:11.194 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: features: layering, exclusive-lock, journaling 2020-04-10T10:06:11.194 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: op_features: 2020-04-10T10:06:11.195 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: flags: 2020-04-10T10:06:11.195 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: create_timestamp: Fri Apr 10 07:09:23 2020 2020-04-10T10:06:11.195 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: access_timestamp: Fri Apr 10 07:09:23 2020 2020-04-10T10:06:11.195 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: modify_timestamp: Fri Apr 10 07:09:23 2020 2020-04-10T10:06:11.195 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: journal: 115186996acb 2020-04-10T10:06:11.195 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring state: enabled 2020-04-10T10:06:11.195 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring mode: snapshot 2020-04-10T10:06:11.196 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring global id: b192e243-6abc-405a-b798-b60b9d74691e 2020-04-10T10:06:11.196 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring primary: true 2020-04-10T10:06:11.198 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.198 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test_force_promote journal status 2020-04-10T10:06:11.198 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:11.198 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test_force_promote journal status 2020-04-10T10:06:11.198 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror --namespace journal status --image test_force_promote 2020-04-10T10:06:11.275 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:minimum_set: 0 2020-04-10T10:06:11.276 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:active_set: 0 2020-04-10T10:06:11.276 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:registered clients: 2020-04-10T10:06:11.276 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: [id=, commit_position=[positions=[[object_number=1, tag_tid=2, entry_tid=101], [object_number=0, tag_tid=2, entry_tid=100], [object_number=3, tag_tid=2, entry_tid=99], [object_number=2, tag_tid=2, entry_tid=98]]], state=connected] 2020-04-10T10:06:11.283 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.284 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster1 mirror rbd_mirroring omap vals 2020-04-10T10:06:11.284 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:11.284 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1 mirror rbd_mirroring omap vals 2020-04-10T10:06:11.284 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rados --cluster cluster1 -p mirror --namespace listomapvals rbd_mirroring 2020-04-10T10:06:11.323 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:global_003078e2-ed6c-4f36-8bc5-ce0c44ef25d5 2020-04-10T10:06:11.323 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (16 bytes) : 2020-04-10T10:06:11.323 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 0c 00 00 00 31 31 33 66 32 30 39 63 63 34 32 39 |....113f209cc429| 2020-04-10T10:06:11.324 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 2020-04-10T10:06:11.324 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.324 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:global_4b1961cb-5406-443f-90f1-3627ac4ffffd 2020-04-10T10:06:11.324 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (16 bytes) : 2020-04-10T10:06:11.324 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 0c 00 00 00 31 33 34 66 64 30 65 39 39 37 30 30 |....134fd0e99700| 2020-04-10T10:06:11.324 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 2020-04-10T10:06:11.324 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.325 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:global_56419286-13e9-49f6-a20d-2abe6ede083e 2020-04-10T10:06:11.325 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (16 bytes) : 2020-04-10T10:06:11.325 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 0c 00 00 00 31 31 33 66 35 61 65 64 30 33 61 30 |....113f5aed03a0| 2020-04-10T10:06:11.325 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 2020-04-10T10:06:11.325 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.325 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:global_70d86755-f73a-4bbe-9493-1b79a191cc0c 2020-04-10T10:06:11.325 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (15 bytes) : 2020-04-10T10:06:11.326 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 0b 00 00 00 31 31 34 62 62 34 62 61 63 35 63 |....114bb4bac5c| 2020-04-10T10:06:11.327 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000000f 2020-04-10T10:06:11.327 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.327 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:global_a8183526-4331-4f0d-af4a-7c93c501e927 2020-04-10T10:06:11.327 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (15 bytes) : 2020-04-10T10:06:11.327 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 0b 00 00 00 31 31 33 66 64 65 37 61 65 61 61 |....113fde7aeaa| 2020-04-10T10:06:11.327 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000000f 2020-04-10T10:06:11.327 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.328 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:global_a9af7d9e-1404-4730-9e18-c0418a915346 2020-04-10T10:06:11.328 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (16 bytes) : 2020-04-10T10:06:11.328 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 0c 00 00 00 31 31 34 62 63 65 65 31 37 39 64 61 |....114bcee179da| 2020-04-10T10:06:11.328 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 2020-04-10T10:06:11.328 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.328 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:global_b192e243-6abc-405a-b798-b60b9d74691e 2020-04-10T10:06:11.328 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (16 bytes) : 2020-04-10T10:06:11.329 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 0c 00 00 00 31 31 35 31 38 36 39 39 36 61 63 62 |....115186996acb| 2020-04-10T10:06:11.329 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 2020-04-10T10:06:11.329 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.329 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:global_b583e294-8b69-4e57-9966-783595c454d6 2020-04-10T10:06:11.329 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (15 bytes) : 2020-04-10T10:06:11.329 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 0b 00 00 00 31 31 34 33 39 30 31 30 62 34 31 |....11439010b41| 2020-04-10T10:06:11.329 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000000f 2020-04-10T10:06:11.330 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.330 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image_113f209cc429 2020-04-10T10:06:11.330 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (48 bytes) : 2020-04-10T10:06:11.330 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 02 01 2a 00 00 00 24 00 00 00 30 30 33 30 37 38 |..*...$...003078| 2020-04-10T10:06:11.330 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 65 32 2d 65 64 36 63 2d 34 66 33 36 2d 38 62 63 |e2-ed6c-4f36-8bc| 2020-04-10T10:06:11.330 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 35 2d 63 65 30 63 34 34 65 66 32 35 64 35 01 01 |5-ce0c44ef25d5..| 2020-04-10T10:06:11.330 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 2020-04-10T10:06:11.331 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.331 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image_113f5aed03a0 2020-04-10T10:06:11.331 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (48 bytes) : 2020-04-10T10:06:11.331 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 02 01 2a 00 00 00 24 00 00 00 35 36 34 31 39 32 |..*...$...564192| 2020-04-10T10:06:11.331 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 38 36 2d 31 33 65 39 2d 34 39 66 36 2d 61 32 30 |86-13e9-49f6-a20| 2020-04-10T10:06:11.331 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 64 2d 32 61 62 65 36 65 64 65 30 38 33 65 01 01 |d-2abe6ede083e..| 2020-04-10T10:06:11.331 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 2020-04-10T10:06:11.331 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.332 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image_113fde7aeaa 2020-04-10T10:06:11.332 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (48 bytes) : 2020-04-10T10:06:11.332 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 02 01 2a 00 00 00 24 00 00 00 61 38 31 38 33 35 |..*...$...a81835| 2020-04-10T10:06:11.332 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 32 36 2d 34 33 33 31 2d 34 66 30 64 2d 61 66 34 |26-4331-4f0d-af4| 2020-04-10T10:06:11.332 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 61 2d 37 63 39 33 63 35 30 31 65 39 32 37 01 01 |a-7c93c501e927..| 2020-04-10T10:06:11.332 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 2020-04-10T10:06:11.332 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.333 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image_11439010b41 2020-04-10T10:06:11.333 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (48 bytes) : 2020-04-10T10:06:11.333 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 02 01 2a 00 00 00 24 00 00 00 62 35 38 33 65 32 |..*...$...b583e2| 2020-04-10T10:06:11.333 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 39 34 2d 38 62 36 39 2d 34 65 35 37 2d 39 39 36 |94-8b69-4e57-996| 2020-04-10T10:06:11.333 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 36 2d 37 38 33 35 39 35 63 34 35 34 64 36 01 01 |6-783595c454d6..| 2020-04-10T10:06:11.333 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 2020-04-10T10:06:11.333 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.334 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image_114bb4bac5c 2020-04-10T10:06:11.334 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (48 bytes) : 2020-04-10T10:06:11.334 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 02 01 2a 00 00 00 24 00 00 00 37 30 64 38 36 37 |..*...$...70d867| 2020-04-10T10:06:11.334 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 35 35 2d 66 37 33 61 2d 34 62 62 65 2d 39 34 39 |55-f73a-4bbe-949| 2020-04-10T10:06:11.334 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 33 2d 31 62 37 39 61 31 39 31 63 63 30 63 01 01 |3-1b79a191cc0c..| 2020-04-10T10:06:11.334 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 2020-04-10T10:06:11.334 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.335 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image_114bcee179da 2020-04-10T10:06:11.335 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (48 bytes) : 2020-04-10T10:06:11.335 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 02 01 2a 00 00 00 24 00 00 00 61 39 61 66 37 64 |..*...$...a9af7d| 2020-04-10T10:06:11.335 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 39 65 2d 31 34 30 34 2d 34 37 33 30 2d 39 65 31 |9e-1404-4730-9e1| 2020-04-10T10:06:11.335 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 38 2d 63 30 34 31 38 61 39 31 35 33 34 36 01 01 |8-c0418a915346..| 2020-04-10T10:06:11.335 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 2020-04-10T10:06:11.335 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.335 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image_115186996acb 2020-04-10T10:06:11.336 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (48 bytes) : 2020-04-10T10:06:11.336 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 02 01 2a 00 00 00 24 00 00 00 62 31 39 32 65 32 |..*...$...b192e2| 2020-04-10T10:06:11.336 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 34 33 2d 36 61 62 63 2d 34 30 35 61 2d 62 37 39 |43-6abc-405a-b79| 2020-04-10T10:06:11.336 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 38 2d 62 36 30 62 39 64 37 34 36 39 31 65 01 01 |8-b60b9d74691e..| 2020-04-10T10:06:11.336 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 2020-04-10T10:06:11.336 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.337 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image_134fd0e99700 2020-04-10T10:06:11.337 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (48 bytes) : 2020-04-10T10:06:11.337 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 02 01 2a 00 00 00 24 00 00 00 34 62 31 39 36 31 |..*...$...4b1961| 2020-04-10T10:06:11.337 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 63 62 2d 35 34 30 36 2d 34 34 33 66 2d 39 30 66 |cb-5406-443f-90f| 2020-04-10T10:06:11.338 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 31 2d 33 36 32 37 61 63 34 66 66 66 66 64 01 01 |1-3627ac4ffffd..| 2020-04-10T10:06:11.338 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 2020-04-10T10:06:11.338 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.338 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:mirror_mode 2020-04-10T10:06:11.338 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (4 bytes) : 2020-04-10T10:06:11.338 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 00 00 00 |....| 2020-04-10T10:06:11.338 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000004 2020-04-10T10:06:11.339 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.339 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:mirror_peer_2571f49c-d767-4112-ae12-d8b71263d58b 2020-04-10T10:06:11.339 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (136 bytes) : 2020-04-10T10:06:11.339 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 02 01 82 00 00 00 24 00 00 00 32 35 37 31 66 34 |......$...2571f4| 2020-04-10T10:06:11.339 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 39 63 2d 64 37 36 37 2d 34 31 31 32 2d 61 65 31 |9c-d767-4112-ae1| 2020-04-10T10:06:11.339 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 32 2d 64 38 62 37 31 32 36 33 64 35 38 62 0c 00 |2-d8b71263d58b..| 2020-04-10T10:06:11.339 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 63 6c 75 73 74 65 72 32 2d 44 4e 45 0d 00 |..cluster2-DNE..| 2020-04-10T10:06:11.340 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 00 00 63 6c 69 65 6e 74 2e 6d 69 72 72 6f 72 ff |..client.mirror.| 2020-04-10T10:06:11.340 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 ff ff ff ff ff ff ff 02 24 00 00 00 38 32 35 33 |........$...8253| 2020-04-10T10:06:11.340 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000060 66 64 65 32 2d 38 36 66 38 2d 34 34 33 61 2d 62 |fde2-86f8-443a-b| 2020-04-10T10:06:11.340 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000070 34 62 66 2d 65 63 37 64 61 65 32 66 61 66 61 33 |4bf-ec7dae2fafa3| 2020-04-10T10:06:11.340 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000080 11 45 90 5e 08 6f a6 17 |.E.^.o..| 2020-04-10T10:06:11.340 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000088 2020-04-10T10:06:11.340 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.341 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:mirror_uuid 2020-04-10T10:06:11.341 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (36 bytes) : 2020-04-10T10:06:11.341 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 36 32 30 64 64 62 30 30 2d 30 66 34 39 2d 34 37 |620ddb00-0f49-47| 2020-04-10T10:06:11.341 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 35 38 2d 61 32 31 36 2d 38 64 39 63 63 38 39 30 |58-a216-8d9cc890| 2020-04-10T10:06:11.341 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 64 30 66 61 |d0fa| 2020-04-10T10:06:11.341 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000024 2020-04-10T10:06:11.342 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.342 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:remote_status_global_003078e2-ed6c-4f36-8bc5-ce0c44ef25d5_8253fde2-86f8-443a-b4bf-ec7dae2fafa3 2020-04-10T10:06:11.342 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (92 bytes) : 2020-04-10T10:06:11.342 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 4e 12 00 00 00 00 00 00 01 |..,....N........| 2020-04-10T10:06:11.343 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 aa d9 82 52 10 00 |.............R..| 2020-04-10T10:06:11.343 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T10:06:11.343 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 24 00 00 00 06 16 00 00 00 6c 6f 63 |....$........loc| 2020-04-10T10:06:11.343 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 61 6c 20 69 6d 61 67 65 20 69 73 20 70 72 69 6d |al image is prim| 2020-04-10T10:06:11.343 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 61 72 79 0e 45 90 5e 61 87 69 1c 00 |ary.E.^a.i..| 2020-04-10T10:06:11.343 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000005c 2020-04-10T10:06:11.343 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.344 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:remote_status_global_3876de2e-b17d-4c86-94c4-90a11916f44e_8253fde2-86f8-443a-b4bf-ec7dae2fafa3 2020-04-10T10:06:11.344 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (96 bytes) : 2020-04-10T10:06:11.344 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 4e 12 00 00 00 00 00 00 01 |..,....N........| 2020-04-10T10:06:11.344 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 aa d9 82 52 10 00 |.............R..| 2020-04-10T10:06:11.344 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T10:06:11.344 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 28 00 00 00 01 1a 00 00 00 65 72 72 |....(........err| 2020-04-10T10:06:11.344 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 6f 72 20 62 6f 6f 74 73 74 72 61 70 70 69 6e 67 |or bootstrapping| 2020-04-10T10:06:11.345 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 20 72 65 70 6c 61 79 39 1c 90 5e 0c 9d b3 03 00 | replay9..^.....| 2020-04-10T10:06:11.345 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000060 2020-04-10T10:06:11.345 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.345 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:remote_status_global_4334771a-4312-4fab-8e12-36d78acc7af1_8253fde2-86f8-443a-b4bf-ec7dae2fafa3 2020-04-10T10:06:11.345 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (96 bytes) : 2020-04-10T10:06:11.345 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 4c 12 00 00 00 00 00 00 01 |..,....L........| 2020-04-10T10:06:11.345 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 a5 ab 32 c8 10 00 |............2...| 2020-04-10T10:06:11.346 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T10:06:11.346 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 28 00 00 00 01 1a 00 00 00 65 72 72 |....(........err| 2020-04-10T10:06:11.346 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 6f 72 20 62 6f 6f 74 73 74 72 61 70 70 69 6e 67 |or bootstrapping| 2020-04-10T10:06:11.346 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 20 72 65 70 6c 61 79 18 1c 90 5e e2 03 d0 39 00 | replay...^...9.| 2020-04-10T10:06:11.346 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000060 2020-04-10T10:06:11.346 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.346 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:remote_status_global_4b1961cb-5406-443f-90f1-3627ac4ffffd_8253fde2-86f8-443a-b4bf-ec7dae2fafa3 2020-04-10T10:06:11.346 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (186 bytes) : 2020-04-10T10:06:11.347 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 4c 12 00 00 00 00 00 00 01 |..,....L........| 2020-04-10T10:06:11.347 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 a5 ab 32 c8 10 00 |............2...| 2020-04-10T10:06:11.347 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T10:06:11.347 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 82 00 00 00 04 74 00 00 00 72 65 70 |.........t...rep| 2020-04-10T10:06:11.347 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 6c 61 79 69 6e 67 2c 20 7b 22 72 65 70 6c 61 79 |laying, {"replay| 2020-04-10T10:06:11.347 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 5f 73 74 61 74 65 22 3a 20 22 69 64 6c 65 22 2c |_state": "idle",| 2020-04-10T10:06:11.347 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000060 20 22 72 65 6d 6f 74 65 5f 73 6e 61 70 73 68 6f | "remote_snapsho| 2020-04-10T10:06:11.348 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000070 74 5f 74 69 6d 65 73 74 61 6d 70 22 3a 20 31 35 |t_timestamp": 15| 2020-04-10T10:06:11.348 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000080 38 36 35 30 32 35 39 38 2c 20 22 6c 6f 63 61 6c |86502598, "local| 2020-04-10T10:06:11.348 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000090 5f 73 6e 61 70 73 68 6f 74 5f 74 69 6d 65 73 74 |_snapshot_timest| 2020-04-10T10:06:11.348 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:000000a0 61 6d 70 22 3a 20 31 35 38 36 35 30 32 35 39 38 |amp": 1586502598| 2020-04-10T10:06:11.348 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:000000b0 7d 64 1c 90 5e 17 e2 8b 09 00 |}d..^.....| 2020-04-10T10:06:11.348 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:000000ba 2020-04-10T10:06:11.348 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.349 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:remote_status_global_56419286-13e9-49f6-a20d-2abe6ede083e_8253fde2-86f8-443a-b4bf-ec7dae2fafa3 2020-04-10T10:06:11.349 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (92 bytes) : 2020-04-10T10:06:11.349 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 4e 12 00 00 00 00 00 00 01 |..,....N........| 2020-04-10T10:06:11.349 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 aa d9 82 52 10 00 |.............R..| 2020-04-10T10:06:11.349 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T10:06:11.349 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 24 00 00 00 06 16 00 00 00 6c 6f 63 |....$........loc| 2020-04-10T10:06:11.349 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 61 6c 20 69 6d 61 67 65 20 69 73 20 70 72 69 6d |al image is prim| 2020-04-10T10:06:11.350 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 61 72 79 0e 45 90 5e c7 83 a2 1c 00 |ary.E.^.....| 2020-04-10T10:06:11.350 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000005c 2020-04-10T10:06:11.350 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.350 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:remote_status_global_5d3e94e2-1c6c-4146-b461-67ecadf338c4_8253fde2-86f8-443a-b4bf-ec7dae2fafa3 2020-04-10T10:06:11.350 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (96 bytes) : 2020-04-10T10:06:11.350 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 4c 12 00 00 00 00 00 00 01 |..,....L........| 2020-04-10T10:06:11.350 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 a5 ab 32 c8 10 00 |............2...| 2020-04-10T10:06:11.351 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T10:06:11.351 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 28 00 00 00 01 1a 00 00 00 65 72 72 |....(........err| 2020-04-10T10:06:11.351 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 6f 72 20 62 6f 6f 74 73 74 72 61 70 70 69 6e 67 |or bootstrapping| 2020-04-10T10:06:11.351 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 20 72 65 70 6c 61 79 18 1c 90 5e 32 84 d2 39 00 | replay...^2..9.| 2020-04-10T10:06:11.351 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000060 2020-04-10T10:06:11.351 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.351 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:remote_status_global_697ea219-b252-4c33-ba20-9b31ea5be82a_8253fde2-86f8-443a-b4bf-ec7dae2fafa3 2020-04-10T10:06:11.352 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (96 bytes) : 2020-04-10T10:06:11.352 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 4e 12 00 00 00 00 00 00 01 |..,....N........| 2020-04-10T10:06:11.352 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 aa d9 82 52 10 00 |.............R..| 2020-04-10T10:06:11.352 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T10:06:11.352 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 28 00 00 00 01 1a 00 00 00 65 72 72 |....(........err| 2020-04-10T10:06:11.353 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 6f 72 20 62 6f 6f 74 73 74 72 61 70 70 69 6e 67 |or bootstrapping| 2020-04-10T10:06:11.353 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 20 72 65 70 6c 61 79 18 1c 90 5e 6d bf 2d 3a 00 | replay...^m.-:.| 2020-04-10T10:06:11.353 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000060 2020-04-10T10:06:11.353 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.353 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:remote_status_global_70d86755-f73a-4bbe-9493-1b79a191cc0c_8253fde2-86f8-443a-b4bf-ec7dae2fafa3 2020-04-10T10:06:11.353 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (92 bytes) : 2020-04-10T10:06:11.353 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 4c 12 00 00 00 00 00 00 01 |..,....L........| 2020-04-10T10:06:11.354 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 a5 ab 32 c8 10 00 |............2...| 2020-04-10T10:06:11.354 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T10:06:11.354 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 24 00 00 00 06 16 00 00 00 6c 6f 63 |....$........loc| 2020-04-10T10:06:11.354 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 61 6c 20 69 6d 61 67 65 20 69 73 20 70 72 69 6d |al image is prim| 2020-04-10T10:06:11.354 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 61 72 79 5f 1c 90 5e dd ca 69 0b 00 |ary_..^..i..| 2020-04-10T10:06:11.354 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000005c 2020-04-10T10:06:11.354 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.355 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:remote_status_global_79225ce2-9203-4752-bc9b-35a11186acdb_8253fde2-86f8-443a-b4bf-ec7dae2fafa3 2020-04-10T10:06:11.355 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (96 bytes) : 2020-04-10T10:06:11.355 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 4e 12 00 00 00 00 00 00 01 |..,....N........| 2020-04-10T10:06:11.355 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 aa d9 82 52 10 00 |.............R..| 2020-04-10T10:06:11.355 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T10:06:11.355 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 28 00 00 00 01 1a 00 00 00 65 72 72 |....(........err| 2020-04-10T10:06:11.355 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 6f 72 20 62 6f 6f 74 73 74 72 61 70 70 69 6e 67 |or bootstrapping| 2020-04-10T10:06:11.355 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 20 72 65 70 6c 61 79 0e 45 90 5e 69 c2 d2 1b 00 | replay.E.^i....| 2020-04-10T10:06:11.356 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000060 2020-04-10T10:06:11.356 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.356 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:remote_status_global_925c4d02-9086-4edd-b4f3-79c1b7e3734c_8253fde2-86f8-443a-b4bf-ec7dae2fafa3 2020-04-10T10:06:11.357 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (96 bytes) : 2020-04-10T10:06:11.357 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 4c 12 00 00 00 00 00 00 01 |..,....L........| 2020-04-10T10:06:11.357 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 a5 ab 32 c8 10 00 |............2...| 2020-04-10T10:06:11.357 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T10:06:11.357 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 28 00 00 00 01 1a 00 00 00 65 72 72 |....(........err| 2020-04-10T10:06:11.357 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 6f 72 20 62 6f 6f 74 73 74 72 61 70 70 69 6e 67 |or bootstrapping| 2020-04-10T10:06:11.358 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 20 72 65 70 6c 61 79 39 1c 90 5e 75 0e fd 03 00 | replay9..^u....| 2020-04-10T10:06:11.358 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000060 2020-04-10T10:06:11.358 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.358 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:remote_status_global_a8183526-4331-4f0d-af4a-7c93c501e927_8253fde2-86f8-443a-b4bf-ec7dae2fafa3 2020-04-10T10:06:11.358 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (92 bytes) : 2020-04-10T10:06:11.358 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 4e 12 00 00 00 00 00 00 01 |..,....N........| 2020-04-10T10:06:11.358 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 aa d9 82 52 10 00 |.............R..| 2020-04-10T10:06:11.359 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T10:06:11.359 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 24 00 00 00 06 16 00 00 00 6c 6f 63 |....$........loc| 2020-04-10T10:06:11.359 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 61 6c 20 69 6d 61 67 65 20 69 73 20 70 72 69 6d |al image is prim| 2020-04-10T10:06:11.359 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 61 72 79 0e 45 90 5e 86 1c a3 1c 00 |ary.E.^.....| 2020-04-10T10:06:11.359 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000005c 2020-04-10T10:06:11.359 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.359 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:remote_status_global_a9af7d9e-1404-4730-9e18-c0418a915346_8253fde2-86f8-443a-b4bf-ec7dae2fafa3 2020-04-10T10:06:11.360 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (92 bytes) : 2020-04-10T10:06:11.360 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 4e 12 00 00 00 00 00 00 01 |..,....N........| 2020-04-10T10:06:11.360 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 aa d9 82 52 10 00 |.............R..| 2020-04-10T10:06:11.360 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T10:06:11.360 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 24 00 00 00 06 16 00 00 00 6c 6f 63 |....$........loc| 2020-04-10T10:06:11.360 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 61 6c 20 69 6d 61 67 65 20 69 73 20 70 72 69 6d |al image is prim| 2020-04-10T10:06:11.360 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 61 72 79 0e 45 90 5e 63 7a a3 1c 00 |ary.E.^cz...| 2020-04-10T10:06:11.361 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000005c 2020-04-10T10:06:11.361 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.361 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:remote_status_global_b192e243-6abc-405a-b798-b60b9d74691e_8253fde2-86f8-443a-b4bf-ec7dae2fafa3 2020-04-10T10:06:11.361 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (92 bytes) : 2020-04-10T10:06:11.361 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 4c 12 00 00 00 00 00 00 01 |..,....L........| 2020-04-10T10:06:11.361 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 a5 ab 32 c8 10 00 |............2...| 2020-04-10T10:06:11.361 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T10:06:11.362 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 24 00 00 00 06 16 00 00 00 6c 6f 63 |....$........loc| 2020-04-10T10:06:11.362 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 61 6c 20 69 6d 61 67 65 20 69 73 20 70 72 69 6d |al image is prim| 2020-04-10T10:06:11.362 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 61 72 79 5f 1c 90 5e 33 70 fa 0a 00 |ary_..^3p...| 2020-04-10T10:06:11.362 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000005c 2020-04-10T10:06:11.362 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.362 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:remote_status_global_b583e294-8b69-4e57-9966-783595c454d6_8253fde2-86f8-443a-b4bf-ec7dae2fafa3 2020-04-10T10:06:11.362 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (92 bytes) : 2020-04-10T10:06:11.363 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 4c 12 00 00 00 00 00 00 01 |..,....L........| 2020-04-10T10:06:11.363 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 a5 ab 32 c8 10 00 |............2...| 2020-04-10T10:06:11.363 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T10:06:11.363 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 24 00 00 00 06 16 00 00 00 6c 6f 63 |....$........loc| 2020-04-10T10:06:11.363 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 61 6c 20 69 6d 61 67 65 20 69 73 20 70 72 69 6d |al image is prim| 2020-04-10T10:06:11.363 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 61 72 79 5f 1c 90 5e 1f 5c 6a 0b 00 |ary_..^.\j..| 2020-04-10T10:06:11.363 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000005c 2020-04-10T10:06:11.364 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.364 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:status_global_003078e2-ed6c-4f36-8bc5-ce0c44ef25d5 2020-04-10T10:06:11.364 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (77 bytes) : 2020-04-10T10:06:11.364 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 3f 11 00 00 00 00 00 00 01 |..,....?........| 2020-04-10T10:06:11.364 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 95 c4 de e4 10 00 |................| 2020-04-10T10:06:11.364 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T10:06:11.364 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 15 00 00 00 06 07 00 00 00 73 74 6f |.............sto| 2020-04-10T10:06:11.365 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 70 70 65 64 12 1c 90 5e 8b 60 e6 17 00 |pped...^.`...| 2020-04-10T10:06:11.365 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000004d 2020-04-10T10:06:11.365 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.365 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:status_global_3876de2e-b17d-4c86-94c4-90a11916f44e 2020-04-10T10:06:11.365 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (96 bytes) : 2020-04-10T10:06:11.365 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 4b 11 00 00 00 00 00 00 01 |..,....K........| 2020-04-10T10:06:11.365 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 c1 9f 21 4e 10 00 |............!N..| 2020-04-10T10:06:11.366 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T10:06:11.366 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 28 00 00 00 01 1a 00 00 00 65 72 72 |....(........err| 2020-04-10T10:06:11.366 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 6f 72 20 62 6f 6f 74 73 74 72 61 70 70 69 6e 67 |or bootstrapping| 2020-04-10T10:06:11.366 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 20 72 65 70 6c 61 79 14 1c 90 5e 31 83 23 21 00 | replay...^1.#!.| 2020-04-10T10:06:11.366 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000060 2020-04-10T10:06:11.366 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.366 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:status_global_4334771a-4312-4fab-8e12-36d78acc7af1 2020-04-10T10:06:11.366 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (96 bytes) : 2020-04-10T10:06:11.367 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 51 11 00 00 00 00 00 00 01 |..,....Q........| 2020-04-10T10:06:11.367 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 60 5f b6 16 10 00 |..........`_....| 2020-04-10T10:06:11.367 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T10:06:11.367 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 28 00 00 00 01 1a 00 00 00 65 72 72 |....(........err| 2020-04-10T10:06:11.367 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 6f 72 20 62 6f 6f 74 73 74 72 61 70 70 69 6e 67 |or bootstrapping| 2020-04-10T10:06:11.367 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 20 72 65 70 6c 61 79 14 1c 90 5e cd 14 5c 21 00 | replay...^..\!.| 2020-04-10T10:06:11.367 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000060 2020-04-10T10:06:11.368 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.368 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:status_global_4b1961cb-5406-443f-90f1-3627ac4ffffd 2020-04-10T10:06:11.368 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (92 bytes) : 2020-04-10T10:06:11.369 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 43 11 00 00 00 00 00 00 01 |..,....C........| 2020-04-10T10:06:11.369 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 09 d2 60 2c 10 00 |............`,..| 2020-04-10T10:06:11.369 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T10:06:11.369 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 24 00 00 00 06 16 00 00 00 6c 6f 63 |....$........loc| 2020-04-10T10:06:11.369 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 61 6c 20 69 6d 61 67 65 20 69 73 20 70 72 69 6d |al image is prim| 2020-04-10T10:06:11.369 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 61 72 79 13 1c 90 5e e4 96 14 1b 00 |ary...^.....| 2020-04-10T10:06:11.370 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000005c 2020-04-10T10:06:11.370 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.370 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:status_global_56419286-13e9-49f6-a20d-2abe6ede083e 2020-04-10T10:06:11.370 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (77 bytes) : 2020-04-10T10:06:11.370 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 3f 11 00 00 00 00 00 00 01 |..,....?........| 2020-04-10T10:06:11.370 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 95 c4 de e4 10 00 |................| 2020-04-10T10:06:11.370 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T10:06:11.371 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 15 00 00 00 06 07 00 00 00 73 74 6f |.............sto| 2020-04-10T10:06:11.371 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 70 70 65 64 12 1c 90 5e 70 41 98 17 00 |pped...^pA...| 2020-04-10T10:06:11.371 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000004d 2020-04-10T10:06:11.371 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.371 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:status_global_5d3e94e2-1c6c-4146-b461-67ecadf338c4 2020-04-10T10:06:11.371 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (96 bytes) : 2020-04-10T10:06:11.372 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 51 11 00 00 00 00 00 00 01 |..,....Q........| 2020-04-10T10:06:11.372 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 60 5f b6 16 10 00 |..........`_....| 2020-04-10T10:06:11.372 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T10:06:11.372 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 28 00 00 00 01 1a 00 00 00 65 72 72 |....(........err| 2020-04-10T10:06:11.372 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 6f 72 20 62 6f 6f 74 73 74 72 61 70 70 69 6e 67 |or bootstrapping| 2020-04-10T10:06:11.372 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 20 72 65 70 6c 61 79 14 1c 90 5e fd 32 81 21 00 | replay...^.2.!.| 2020-04-10T10:06:11.373 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000060 2020-04-10T10:06:11.373 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.373 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:status_global_687e9f7d-ae46-4324-b8f7-7c16605e0fa6 2020-04-10T10:06:11.373 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (96 bytes) : 2020-04-10T10:06:11.373 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 43 11 00 00 00 00 00 00 01 |..,....C........| 2020-04-10T10:06:11.373 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 09 d2 60 2c 10 00 |............`,..| 2020-04-10T10:06:11.373 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T10:06:11.373 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 28 00 00 00 01 1a 00 00 00 65 72 72 |....(........err| 2020-04-10T10:06:11.374 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 6f 72 20 62 6f 6f 74 73 74 72 61 70 70 69 6e 67 |or bootstrapping| 2020-04-10T10:06:11.374 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 20 72 65 70 6c 61 79 13 1c 90 5e ed 3e 4c 1a 00 | replay...^.>L..| 2020-04-10T10:06:11.374 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000060 2020-04-10T10:06:11.374 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.374 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:status_global_697ea219-b252-4c33-ba20-9b31ea5be82a 2020-04-10T10:06:11.375 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (96 bytes) : 2020-04-10T10:06:11.375 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 3f 11 00 00 00 00 00 00 01 |..,....?........| 2020-04-10T10:06:11.375 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 95 c4 de e4 10 00 |................| 2020-04-10T10:06:11.375 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T10:06:11.375 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 28 00 00 00 01 1a 00 00 00 65 72 72 |....(........err| 2020-04-10T10:06:11.375 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 6f 72 20 62 6f 6f 74 73 74 72 61 70 70 69 6e 67 |or bootstrapping| 2020-04-10T10:06:11.375 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 20 72 65 70 6c 61 79 0d 1c 90 5e 2e 8e 6b 16 00 | replay...^..k..| 2020-04-10T10:06:11.375 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000060 2020-04-10T10:06:11.376 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.376 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:status_global_70d86755-f73a-4bbe-9493-1b79a191cc0c 2020-04-10T10:06:11.376 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (77 bytes) : 2020-04-10T10:06:11.376 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 4b 11 00 00 00 00 00 00 01 |..,....K........| 2020-04-10T10:06:11.376 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 c1 9f 21 4e 10 00 |............!N..| 2020-04-10T10:06:11.376 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T10:06:11.376 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 15 00 00 00 06 07 00 00 00 73 74 6f |.............sto| 2020-04-10T10:06:11.377 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 70 70 65 64 14 1c 90 5e 3e d4 e9 26 00 |pped...^>..&.| 2020-04-10T10:06:11.377 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000004d 2020-04-10T10:06:11.377 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.377 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:status_global_79225ce2-9203-4752-bc9b-35a11186acdb 2020-04-10T10:06:11.377 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (97 bytes) : 2020-04-10T10:06:11.377 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 4b 11 00 00 00 00 00 00 01 |..,....K........| 2020-04-10T10:06:11.377 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 c1 9f 21 4e 10 00 |............!N..| 2020-04-10T10:06:11.378 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T10:06:11.378 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 29 00 00 00 00 1b 00 00 00 72 65 6d |....)........rem| 2020-04-10T10:06:11.378 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 6f 74 65 20 69 6d 61 67 65 20 69 73 20 6e 6f 6e |ote image is non| 2020-04-10T10:06:11.378 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 2d 70 72 69 6d 61 72 79 14 1c 90 5e 79 14 9d 21 |-primary...^y..!| 2020-04-10T10:06:11.378 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000060 00 |.| 2020-04-10T10:06:11.378 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000061 2020-04-10T10:06:11.378 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.379 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:status_global_925c4d02-9086-4edd-b4f3-79c1b7e3734c 2020-04-10T10:06:11.379 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (96 bytes) : 2020-04-10T10:06:11.379 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 43 11 00 00 00 00 00 00 01 |..,....C........| 2020-04-10T10:06:11.379 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 09 d2 60 2c 10 00 |............`,..| 2020-04-10T10:06:11.379 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T10:06:11.380 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 28 00 00 00 01 1a 00 00 00 65 72 72 |....(........err| 2020-04-10T10:06:11.380 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 6f 72 20 62 6f 6f 74 73 74 72 61 70 70 69 6e 67 |or bootstrapping| 2020-04-10T10:06:11.380 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 20 72 65 70 6c 61 79 13 1c 90 5e 26 cb 4c 1a 00 | replay...^&.L..| 2020-04-10T10:06:11.380 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000060 2020-04-10T10:06:11.380 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.380 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:status_global_a8183526-4331-4f0d-af4a-7c93c501e927 2020-04-10T10:06:11.381 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (77 bytes) : 2020-04-10T10:06:11.381 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 3f 11 00 00 00 00 00 00 01 |..,....?........| 2020-04-10T10:06:11.381 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 95 c4 de e4 10 00 |................| 2020-04-10T10:06:11.381 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T10:06:11.381 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 15 00 00 00 06 07 00 00 00 73 74 6f |.............sto| 2020-04-10T10:06:11.381 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 70 70 65 64 12 1c 90 5e 06 82 04 18 00 |pped...^.....| 2020-04-10T10:06:11.381 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000004d 2020-04-10T10:06:11.381 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.382 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:status_global_a9af7d9e-1404-4730-9e18-c0418a915346 2020-04-10T10:06:11.382 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (77 bytes) : 2020-04-10T10:06:11.382 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 4b 11 00 00 00 00 00 00 01 |..,....K........| 2020-04-10T10:06:11.382 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 c1 9f 21 4e 10 00 |............!N..| 2020-04-10T10:06:11.382 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T10:06:11.382 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 15 00 00 00 06 07 00 00 00 73 74 6f |.............sto| 2020-04-10T10:06:11.382 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 70 70 65 64 14 1c 90 5e aa 65 77 26 00 |pped...^.ew&.| 2020-04-10T10:06:11.383 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000004d 2020-04-10T10:06:11.383 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.383 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:status_global_b192e243-6abc-405a-b798-b60b9d74691e 2020-04-10T10:06:11.383 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (92 bytes) : 2020-04-10T10:06:11.383 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 51 11 00 00 00 00 00 00 01 |..,....Q........| 2020-04-10T10:06:11.383 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 60 5f b6 16 10 00 |..........`_....| 2020-04-10T10:06:11.383 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T10:06:11.384 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 24 00 00 00 06 16 00 00 00 6c 6f 63 |....$........loc| 2020-04-10T10:06:11.384 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 61 6c 20 69 6d 61 67 65 20 69 73 20 70 72 69 6d |al image is prim| 2020-04-10T10:06:11.384 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 61 72 79 14 1c 90 5e 79 8e fe 21 00 |ary...^y..!.| 2020-04-10T10:06:11.384 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000005c 2020-04-10T10:06:11.384 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.384 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:status_global_b583e294-8b69-4e57-9966-783595c454d6 2020-04-10T10:06:11.385 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (77 bytes) : 2020-04-10T10:06:11.385 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 43 11 00 00 00 00 00 00 01 |..,....C........| 2020-04-10T10:06:11.385 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 09 d2 60 2c 10 00 |............`,..| 2020-04-10T10:06:11.385 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T10:06:11.385 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 15 00 00 00 06 07 00 00 00 73 74 6f |.............sto| 2020-04-10T10:06:11.385 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 70 70 65 64 13 1c 90 5e 52 c6 7b 1f 00 |pped...^R.{..| 2020-04-10T10:06:11.386 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000004d 2020-04-10T10:06:11.386 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.387 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:11.387 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1 mirror ns1 images 2020-04-10T10:06:11.387 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror --namespace ns1 ls -l 2020-04-10T10:06:11.387 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:11.387 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1 mirrorns1 mirror pool info 2020-04-10T10:06:11.388 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror --namespace ns1 mirror pool info 2020-04-10T10:06:11.388 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.388 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster1 mirror ns1 images 2020-04-10T10:06:11.388 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.388 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster1 mirrorns1 mirror pool info 2020-04-10T10:06:11.411 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Mode: image 2020-04-10T10:06:11.415 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.416 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster1 mirrorns1 mirror pool status 2020-04-10T10:06:11.416 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:11.416 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1 mirrorns1 mirror pool status 2020-04-10T10:06:11.416 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster1 -p mirror --namespace ns1 mirror pool status --verbose 2020-04-10T10:06:11.458 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:health: UNKNOWN 2020-04-10T10:06:11.459 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:daemon health: UNKNOWN 2020-04-10T10:06:11.459 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image health: OK 2020-04-10T10:06:11.459 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:images: 0 total 2020-04-10T10:06:11.459 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.459 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:DAEMONS 2020-04-10T10:06:11.459 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: none 2020-04-10T10:06:11.459 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.471 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:11.471 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror --namespace ns1 ls 2020-04-10T10:06:11.471 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:IMAGES 2020-04-10T10:06:11.505 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster1 mirror ns1 rbd_mirroring omap vals 2020-04-10T10:06:11.506 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1 mirror ns1 rbd_mirroring omap vals 2020-04-10T10:06:11.506 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rados --cluster cluster1 -p mirror --namespace ns1 listomapvals rbd_mirroring 2020-04-10T10:06:11.543 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:mirror_mode 2020-04-10T10:06:11.544 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (4 bytes) : 2020-04-10T10:06:11.544 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 00 00 00 |....| 2020-04-10T10:06:11.544 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000004 2020-04-10T10:06:11.544 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.544 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:mirror_uuid 2020-04-10T10:06:11.544 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (36 bytes) : 2020-04-10T10:06:11.544 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 36 66 32 30 33 39 32 30 2d 36 30 39 39 2d 34 32 |6f203920-6099-42| 2020-04-10T10:06:11.545 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 33 63 2d 61 37 64 64 2d 65 61 35 35 31 32 39 36 |3c-a7dd-ea551296| 2020-04-10T10:06:11.545 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 64 39 31 61 |d91a| 2020-04-10T10:06:11.545 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000024 2020-04-10T10:06:11.545 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.548 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.548 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster1 mirror ns2 images 2020-04-10T10:06:11.548 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:11.548 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1 mirror ns2 images 2020-04-10T10:06:11.548 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror --namespace ns2 ls -l 2020-04-10T10:06:11.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:11.589 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1 mirrorns2 mirror pool info 2020-04-10T10:06:11.590 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror --namespace ns2 mirror pool info 2020-04-10T10:06:11.590 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.590 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster1 mirrorns2 mirror pool info 2020-04-10T10:06:11.626 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Mode: image 2020-04-10T10:06:11.628 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.629 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster1 mirrorns2 mirror pool status 2020-04-10T10:06:11.629 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:11.629 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1 mirrorns2 mirror pool status 2020-04-10T10:06:11.629 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster1 -p mirror --namespace ns2 mirror pool status --verbose 2020-04-10T10:06:11.665 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:health: UNKNOWN 2020-04-10T10:06:11.665 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:daemon health: UNKNOWN 2020-04-10T10:06:11.666 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image health: OK 2020-04-10T10:06:11.666 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:images: 0 total 2020-04-10T10:06:11.666 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.666 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:DAEMONS 2020-04-10T10:06:11.666 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: none 2020-04-10T10:06:11.666 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.675 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:IMAGES 2020-04-10T10:06:11.675 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:11.675 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror --namespace ns2 ls 2020-04-10T10:06:11.719 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1 mirror ns2 rbd_mirroring omap vals 2020-04-10T10:06:11.719 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rados --cluster cluster1 -p mirror --namespace ns2 listomapvals rbd_mirroring 2020-04-10T10:06:11.719 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster1 mirror ns2 rbd_mirroring omap vals 2020-04-10T10:06:11.750 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:mirror_mode 2020-04-10T10:06:11.750 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (4 bytes) : 2020-04-10T10:06:11.750 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 00 00 00 |....| 2020-04-10T10:06:11.750 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000004 2020-04-10T10:06:11.750 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.750 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:mirror_uuid 2020-04-10T10:06:11.751 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (36 bytes) : 2020-04-10T10:06:11.751 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 66 32 36 36 31 30 61 64 2d 66 65 38 32 2d 34 66 |f26610ad-fe82-4f| 2020-04-10T10:06:11.751 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 35 30 2d 38 34 37 30 2d 32 61 35 35 31 31 31 61 |50-8470-2a55111a| 2020-04-10T10:06:11.751 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 38 64 61 63 |8dac| 2020-04-10T10:06:11.751 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000024 2020-04-10T10:06:11.751 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.752 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:11.753 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1 mirror_parent images 2020-04-10T10:06:11.753 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror_parent --namespace ls -l 2020-04-10T10:06:11.753 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.753 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster1 mirror_parent images 2020-04-10T10:06:11.819 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:NAME SIZE PARENT FMT PROT LOCK 2020-04-10T10:06:11.819 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test_parent 128 MiB 2 2020-04-10T10:06:11.819 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test_parent@snap 128 MiB 2 yes 2020-04-10T10:06:11.822 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.822 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster1 mirror_parent mirror pool info 2020-04-10T10:06:11.823 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:11.823 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1 mirror_parent mirror pool info 2020-04-10T10:06:11.823 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror_parent --namespace mirror pool info 2020-04-10T10:06:11.862 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Mode: image 2020-04-10T10:06:11.863 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Site Name: cluster1-DNE 2020-04-10T10:06:11.863 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.863 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Peer Sites: 2020-04-10T10:06:11.863 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.863 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:UUID: 467a3104-15d9-4d31-90d5-8005aa7c9eeb 2020-04-10T10:06:11.863 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Name: cluster2-DNE 2020-04-10T10:06:11.864 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Mirror UUID: e151661d-9adc-4448-8574-890d2847c790 2020-04-10T10:06:11.864 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Direction: rx-tx 2020-04-10T10:06:11.864 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Client: client.mirror 2020-04-10T10:06:11.865 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:11.865 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1 mirror_parent mirror pool status 2020-04-10T10:06:11.866 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster1 -p mirror_parent --namespace mirror pool status --verbose 2020-04-10T10:06:11.866 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.866 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster1 mirror_parent mirror pool status 2020-04-10T10:06:11.907 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:health: UNKNOWN 2020-04-10T10:06:11.907 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:daemon health: UNKNOWN 2020-04-10T10:06:11.907 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image health: OK 2020-04-10T10:06:11.907 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:images: 1 total 2020-04-10T10:06:11.907 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 1 stopped 2020-04-10T10:06:11.907 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.908 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:DAEMONS 2020-04-10T10:06:11.908 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: none 2020-04-10T10:06:11.908 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.947 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:IMAGES 2020-04-10T10:06:11.947 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test_parent: 2020-04-10T10:06:11.947 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: global_id: dd523e97-79dd-49a7-b2c9-df255e9660d4 2020-04-10T10:06:11.947 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: down+stopped 2020-04-10T10:06:11.948 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: stopped 2020-04-10T10:06:11.948 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 07:11:14 2020-04-10T10:06:11.948 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: peer_sites: 2020-04-10T10:06:11.948 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: name: cluster2-DNE 2020-04-10T10:06:11.948 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+stopped 2020-04-10T10:06:11.948 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: local image is primary 2020-04-10T10:06:11.948 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 10:06:11 2020-04-10T10:06:11.951 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:11.952 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:11.952 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror_parent --namespace ls 2020-04-10T10:06:11.994 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test_parent info 2020-04-10T10:06:11.994 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test_parent info 2020-04-10T10:06:11.995 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror_parent --namespace info test_parent 2020-04-10T10:06:12.062 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:rbd image 'test_parent': 2020-04-10T10:06:12.062 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: size 128 MiB in 32 objects 2020-04-10T10:06:12.062 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: order 22 (4 MiB objects) 2020-04-10T10:06:12.062 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: snapshot_count: 7 2020-04-10T10:06:12.062 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: id: 1154c59c0497 2020-04-10T10:06:12.062 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: block_name_prefix: rbd_data.1154c59c0497 2020-04-10T10:06:12.063 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: format: 2 2020-04-10T10:06:12.063 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: features: layering, exclusive-lock, journaling, non-primary 2020-04-10T10:06:12.063 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: op_features: 2020-04-10T10:06:12.063 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: flags: 2020-04-10T10:06:12.063 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: create_timestamp: Fri Apr 10 07:09:45 2020 2020-04-10T10:06:12.063 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: access_timestamp: Fri Apr 10 07:09:45 2020 2020-04-10T10:06:12.063 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: modify_timestamp: Fri Apr 10 07:09:45 2020 2020-04-10T10:06:12.064 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: journal: 1154c59c0497 2020-04-10T10:06:12.064 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring state: enabled 2020-04-10T10:06:12.064 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring mode: snapshot 2020-04-10T10:06:12.064 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring global id: dd523e97-79dd-49a7-b2c9-df255e9660d4 2020-04-10T10:06:12.064 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring primary: false 2020-04-10T10:06:12.066 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:12.066 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test_parent journal status 2020-04-10T10:06:12.066 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror_parent --namespace journal status --image test_parent 2020-04-10T10:06:12.066 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:12.067 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test_parent journal status 2020-04-10T10:06:12.140 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:minimum_set: 0 2020-04-10T10:06:12.141 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:active_set: 0 2020-04-10T10:06:12.141 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:registered clients: 2020-04-10T10:06:12.141 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: [id=, commit_position=[positions=[]], state=connected] 2020-04-10T10:06:12.145 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:12.145 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster1 mirror_parent rbd_mirroring omap vals 2020-04-10T10:06:12.145 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:12.146 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1 mirror_parent rbd_mirroring omap vals 2020-04-10T10:06:12.146 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rados --cluster cluster1 -p mirror_parent --namespace listomapvals rbd_mirroring 2020-04-10T10:06:12.186 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:global_dd523e97-79dd-49a7-b2c9-df255e9660d4 2020-04-10T10:06:12.187 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (16 bytes) : 2020-04-10T10:06:12.187 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 0c 00 00 00 31 31 35 34 63 35 39 63 30 34 39 37 |....1154c59c0497| 2020-04-10T10:06:12.187 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 2020-04-10T10:06:12.187 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:12.187 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image_1154c59c0497 2020-04-10T10:06:12.187 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (48 bytes) : 2020-04-10T10:06:12.187 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 02 01 2a 00 00 00 24 00 00 00 64 64 35 32 33 65 |..*...$...dd523e| 2020-04-10T10:06:12.188 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 39 37 2d 37 39 64 64 2d 34 39 61 37 2d 62 32 63 |97-79dd-49a7-b2c| 2020-04-10T10:06:12.188 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 39 2d 64 66 32 35 35 65 39 36 36 30 64 34 01 01 |9-df255e9660d4..| 2020-04-10T10:06:12.188 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 2020-04-10T10:06:12.188 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:12.188 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:mirror_mode 2020-04-10T10:06:12.188 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (4 bytes) : 2020-04-10T10:06:12.188 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 00 00 00 |....| 2020-04-10T10:06:12.189 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000004 2020-04-10T10:06:12.189 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:12.189 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:mirror_peer_467a3104-15d9-4d31-90d5-8005aa7c9eeb 2020-04-10T10:06:12.189 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (136 bytes) : 2020-04-10T10:06:12.189 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 02 01 82 00 00 00 24 00 00 00 34 36 37 61 33 31 |......$...467a31| 2020-04-10T10:06:12.189 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 30 34 2d 31 35 64 39 2d 34 64 33 31 2d 39 30 64 |04-15d9-4d31-90d| 2020-04-10T10:06:12.189 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 35 2d 38 30 30 35 61 61 37 63 39 65 65 62 0c 00 |5-8005aa7c9eeb..| 2020-04-10T10:06:12.190 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 63 6c 75 73 74 65 72 32 2d 44 4e 45 0d 00 |..cluster2-DNE..| 2020-04-10T10:06:12.190 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 00 00 63 6c 69 65 6e 74 2e 6d 69 72 72 6f 72 ff |..client.mirror.| 2020-04-10T10:06:12.190 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 ff ff ff ff ff ff ff 02 24 00 00 00 65 31 35 31 |........$...e151| 2020-04-10T10:06:12.190 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000060 36 36 31 64 2d 39 61 64 63 2d 34 34 34 38 2d 38 |661d-9adc-4448-8| 2020-04-10T10:06:12.190 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000070 35 37 34 2d 38 39 30 64 32 38 34 37 63 37 39 30 |574-890d2847c790| 2020-04-10T10:06:12.190 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000080 11 45 90 5e 4d 03 6c 0d |.E.^M.l.| 2020-04-10T10:06:12.190 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000088 2020-04-10T10:06:12.191 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:12.191 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:mirror_uuid 2020-04-10T10:06:12.191 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (36 bytes) : 2020-04-10T10:06:12.191 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 66 61 35 36 37 64 39 33 2d 66 34 36 35 2d 34 37 |fa567d93-f465-47| 2020-04-10T10:06:12.191 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 36 34 2d 39 33 63 37 2d 31 32 39 64 39 39 33 63 |64-93c7-129d993c| 2020-04-10T10:06:12.191 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 66 38 61 31 |f8a1| 2020-04-10T10:06:12.191 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000024 2020-04-10T10:06:12.192 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:12.192 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:remote_status_global_dd523e97-79dd-49a7-b2c9-df255e9660d4_e151661d-9adc-4448-8574-890d2847c790 2020-04-10T10:06:12.192 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (92 bytes) : 2020-04-10T10:06:12.192 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 54 12 00 00 00 00 00 00 01 |..,....T........| 2020-04-10T10:06:12.192 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 60 22 62 98 10 00 |..........`"b...| 2020-04-10T10:06:12.192 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T10:06:12.193 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 24 00 00 00 06 16 00 00 00 6c 6f 63 |....$........loc| 2020-04-10T10:06:12.193 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 61 6c 20 69 6d 61 67 65 20 69 73 20 70 72 69 6d |al image is prim| 2020-04-10T10:06:12.193 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 61 72 79 13 45 90 5e 95 a6 47 23 00 |ary.E.^..G#.| 2020-04-10T10:06:12.193 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000005c 2020-04-10T10:06:12.193 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:12.193 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:status_global_dd523e97-79dd-49a7-b2c9-df255e9660d4 2020-04-10T10:06:12.193 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (77 bytes) : 2020-04-10T10:06:12.194 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 54 11 00 00 00 00 00 00 01 |..,....T........| 2020-04-10T10:06:12.194 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 82 7b f2 0b 10 00 |...........{....| 2020-04-10T10:06:12.194 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T10:06:12.194 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 15 00 00 00 06 07 00 00 00 73 74 6f |.............sto| 2020-04-10T10:06:12.194 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 70 70 65 64 12 1c 90 5e 3d 19 b6 14 00 |pped...^=....| 2020-04-10T10:06:12.194 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000004d 2020-04-10T10:06:12.195 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:12.195 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:12.196 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster1 mirror_parent ns1 images 2020-04-10T10:06:12.196 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:12.196 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1 mirror_parent ns1 images 2020-04-10T10:06:12.196 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror_parent --namespace ns1 ls -l 2020-04-10T10:06:12.223 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: namespace 'ns1' does not exist. 2020-04-10T10:06:12.224 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: listing images failed: (2) No such file or directory 2020-04-10T10:06:12.232 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:12.233 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster1 mirror_parentns1 mirror pool info 2020-04-10T10:06:12.233 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:12.233 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1 mirror_parentns1 mirror pool info 2020-04-10T10:06:12.233 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror_parent --namespace ns1 mirror pool info 2020-04-10T10:06:12.268 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: namespace 'ns1' does not exist. 2020-04-10T10:06:12.270 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:12.270 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster1 mirror_parentns1 mirror pool status 2020-04-10T10:06:12.270 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:12.271 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1 mirror_parentns1 mirror pool status 2020-04-10T10:06:12.271 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster1 -p mirror_parent --namespace ns1 mirror pool status --verbose 2020-04-10T10:06:12.308 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: namespace 'ns1' does not exist. 2020-04-10T10:06:12.311 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:12.311 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:12.311 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror_parent --namespace ns1 ls 2020-04-10T10:06:12.351 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1 mirror_parent ns1 rbd_mirroring omap vals 2020-04-10T10:06:12.351 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rados --cluster cluster1 -p mirror_parent --namespace ns1 listomapvals rbd_mirroring 2020-04-10T10:06:12.351 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster1 mirror_parent ns1 rbd_mirroring omap vals 2020-04-10T10:06:12.392 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:error getting omap keys mirror_parent/rbd_mirroring: (2) No such file or directory 2020-04-10T10:06:12.394 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:12.394 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster1 mirror_parent ns2 images 2020-04-10T10:06:12.395 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:12.395 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1 mirror_parent ns2 images 2020-04-10T10:06:12.395 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror_parent --namespace ns2 ls -l 2020-04-10T10:06:12.636 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: namespace 'ns2' does not exist. 2020-04-10T10:06:12.636 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: listing images failed: (2) No such file or directory 2020-04-10T10:06:12.644 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:12.644 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1 mirror_parentns2 mirror pool info 2020-04-10T10:06:12.644 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror_parent --namespace ns2 mirror pool info 2020-04-10T10:06:12.644 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:12.645 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster1 mirror_parentns2 mirror pool info 2020-04-10T10:06:12.679 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: namespace 'ns2' does not exist. 2020-04-10T10:06:12.691 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:12.692 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster1 mirror_parentns2 mirror pool status 2020-04-10T10:06:12.692 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:12.692 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1 mirror_parentns2 mirror pool status 2020-04-10T10:06:12.692 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster1 -p mirror_parent --namespace ns2 mirror pool status --verbose 2020-04-10T10:06:12.724 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: namespace 'ns2' does not exist. 2020-04-10T10:06:12.725 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:12.725 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster1 -p mirror_parent --namespace ns2 ls 2020-04-10T10:06:12.725 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:12.761 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster1 mirror_parent ns2 rbd_mirroring omap vals 2020-04-10T10:06:12.761 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1 mirror_parent ns2 rbd_mirroring omap vals 2020-04-10T10:06:12.762 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rados --cluster cluster1 -p mirror_parent --namespace ns2 listomapvals rbd_mirroring 2020-04-10T10:06:12.799 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:error getting omap keys mirror_parent/rbd_mirroring: (2) No such file or directory 2020-04-10T10:06:12.801 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:12.801 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2 status 2020-04-10T10:06:12.801 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= ceph --cluster cluster2 -s 2020-04-10T10:06:12.801 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:12.802 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster2 status 2020-04-10T10:06:13.211 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: cluster: 2020-04-10T10:06:13.211 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: id: b4476b22-7943-481f-a1ec-34bc78d4adbc 2020-04-10T10:06:13.211 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: health: HEALTH_OK 2020-04-10T10:06:13.211 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:13.211 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: services: 2020-04-10T10:06:13.211 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mon: 1 daemons, quorum a (age 3h) 2020-04-10T10:06:13.212 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mgr: x(active, since 3h) 2020-04-10T10:06:13.212 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: osd: 3 osds: 3 up (since 3h), 3 in (since 3h) 2020-04-10T10:06:13.212 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: rbd-mirror: 2 daemons active (4491, 4499) 2020-04-10T10:06:13.212 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:13.212 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: task status: 2020-04-10T10:06:13.212 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:13.213 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: data: 2020-04-10T10:06:13.213 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: pools: 4 pools, 137 pgs 2020-04-10T10:06:13.213 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: objects: 574 objects, 825 MiB 2020-04-10T10:06:13.213 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: usage: 3.0 GiB used, 267 GiB / 270 GiB avail 2020-04-10T10:06:13.213 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: pgs: 137 active+clean 2020-04-10T10:06:13.213 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:13.213 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: io: 2020-04-10T10:06:13.213 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: client: 9.2 KiB/s rd, 1023 B/s wr, 9 op/s rd, 0 op/s wr 2020-04-10T10:06:13.214 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:13.222 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= ceph --cluster cluster2 service dump 2020-04-10T10:06:13.546 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:{ 2020-04-10T10:06:13.546 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "epoch": 4, 2020-04-10T10:06:13.547 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "modified": "2020-04-10T07:11:21.915914+0000", 2020-04-10T10:06:13.547 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "services": { 2020-04-10T10:06:13.547 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "rbd-mirror": { 2020-04-10T10:06:13.547 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "daemons": { 2020-04-10T10:06:13.547 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "summary": "", 2020-04-10T10:06:13.547 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "4491": { 2020-04-10T10:06:13.547 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "start_epoch": 2, 2020-04-10T10:06:13.548 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "start_stamp": "2020-04-10T07:08:06.046076+0000", 2020-04-10T10:06:13.548 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "gid": 4491, 2020-04-10T10:06:13.548 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "addr": "172.21.15.204:0/4237060419", 2020-04-10T10:06:13.548 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "metadata": { 2020-04-10T10:06:13.548 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "arch": "x86_64", 2020-04-10T10:06:13.548 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "ceph_release": "octopus", 2020-04-10T10:06:13.549 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "ceph_version": "ceph version 15.2.0-148-g39357d2 (39357d2a782a02d1ea09c76cbe7bc6f222a1127a) octopus (stable)", 2020-04-10T10:06:13.549 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "ceph_version_short": "15.2.0-148-g39357d2", 2020-04-10T10:06:13.549 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz", 2020-04-10T10:06:13.549 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "distro": "ubuntu", 2020-04-10T10:06:13.549 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "distro_description": "Ubuntu 18.04.4 LTS", 2020-04-10T10:06:13.549 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "distro_version": "18.04", 2020-04-10T10:06:13.549 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "hostname": "smithi204", 2020-04-10T10:06:13.550 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "id": "mirror.2", 2020-04-10T10:06:13.550 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "instance_id": "4491", 2020-04-10T10:06:13.550 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "kernel_description": "#97-Ubuntu SMP Wed Apr 1 03:25:46 UTC 2020", 2020-04-10T10:06:13.550 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "kernel_version": "4.15.0-96-generic", 2020-04-10T10:06:13.550 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "mem_swap_kb": "2097148", 2020-04-10T10:06:13.550 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "mem_total_kb": "32823944", 2020-04-10T10:06:13.550 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "os": "Linux" 2020-04-10T10:06:13.551 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: }, 2020-04-10T10:06:13.551 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "task_status": {} 2020-04-10T10:06:13.551 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: }, 2020-04-10T10:06:13.551 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "4499": { 2020-04-10T10:06:13.551 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "start_epoch": 2, 2020-04-10T10:06:13.551 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "start_stamp": "2020-04-10T07:08:06.075848+0000", 2020-04-10T10:06:13.551 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "gid": 4499, 2020-04-10T10:06:13.552 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "addr": "172.21.15.204:0/2107014639", 2020-04-10T10:06:13.552 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "metadata": { 2020-04-10T10:06:13.552 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "arch": "x86_64", 2020-04-10T10:06:13.552 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "ceph_release": "octopus", 2020-04-10T10:06:13.552 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "ceph_version": "ceph version 15.2.0-148-g39357d2 (39357d2a782a02d1ea09c76cbe7bc6f222a1127a) octopus (stable)", 2020-04-10T10:06:13.552 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "ceph_version_short": "15.2.0-148-g39357d2", 2020-04-10T10:06:13.552 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz", 2020-04-10T10:06:13.553 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "distro": "ubuntu", 2020-04-10T10:06:13.553 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "distro_description": "Ubuntu 18.04.4 LTS", 2020-04-10T10:06:13.553 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "distro_version": "18.04", 2020-04-10T10:06:13.553 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "hostname": "smithi204", 2020-04-10T10:06:13.553 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "id": "mirror.3", 2020-04-10T10:06:13.553 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "instance_id": "4499", 2020-04-10T10:06:13.553 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "kernel_description": "#97-Ubuntu SMP Wed Apr 1 03:25:46 UTC 2020", 2020-04-10T10:06:13.554 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "kernel_version": "4.15.0-96-generic", 2020-04-10T10:06:13.554 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "mem_swap_kb": "2097148", 2020-04-10T10:06:13.554 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "mem_total_kb": "32823944", 2020-04-10T10:06:13.554 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "os": "Linux" 2020-04-10T10:06:13.554 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: }, 2020-04-10T10:06:13.554 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "task_status": {} 2020-04-10T10:06:13.554 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: } 2020-04-10T10:06:13.555 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: } 2020-04-10T10:06:13.555 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: } 2020-04-10T10:06:13.555 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: } 2020-04-10T10:06:13.555 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:} 2020-04-10T10:06:13.557 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= ceph --cluster cluster2 service status 2020-04-10T10:06:13.889 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:{ 2020-04-10T10:06:13.889 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "rbd-mirror": { 2020-04-10T10:06:13.889 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "4491": { 2020-04-10T10:06:13.889 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "status_stamp": "2020-04-10T09:56:21.105849+0000", 2020-04-10T10:06:13.889 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "last_beacon": "2020-04-10T10:06:11.128866+0000", 2020-04-10T10:06:13.890 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "status": { 2020-04-10T10:06:13.890 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "json": "{\"3\":{\"name\":\"mirror\",\"callouts\":{},\"image_assigned_count\":4,\"image_error_count\":0,\"image_warning_count\":0,\"instance_id\":\"4502\",\"namespaces\":{\"ns1\":{\"image_assigned_count\":0,\"image_error_count\":0,\"image_warning_count\":0},\"ns2\":{\"image_assigned_count\":0,\"image_error_count\":0,\"image_warning_count\":0}}},\"4\":{\"name\":\"mirror_parent\",\"callouts\":{}}}" 2020-04-10T10:06:13.890 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: } 2020-04-10T10:06:13.890 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: }, 2020-04-10T10:06:13.890 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "4499": { 2020-04-10T10:06:13.890 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "status_stamp": "2020-04-10T10:06:11.184011+0000", 2020-04-10T10:06:13.890 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "last_beacon": "2020-04-10T10:06:11.184011+0000", 2020-04-10T10:06:13.891 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "status": { 2020-04-10T10:06:13.891 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: "json": "{\"3\":{\"name\":\"mirror\",\"callouts\":{},\"image_assigned_count\":5,\"image_error_count\":1,\"image_local_count\":9,\"image_remote_count\":8,\"image_warning_count\":0,\"instance_id\":\"4510\",\"leader\":true,\"namespaces\":{\"ns1\":{\"image_assigned_count\":0,\"image_error_count\":0,\"image_local_count\":0,\"image_remote_count\":0,\"image_warning_count\":0},\"ns2\":{\"image_assigned_count\":0,\"image_error_count\":0,\"image_local_count\":0,\"image_remote_count\":0,\"image_warning_count\":0}}},\"4\":{\"name\":\"mirror_parent\",\"callouts\":{},\"image_assigned_count\":1,\"image_error_count\":0,\"image_local_count\":1,\"image_remote_count\":1,\"image_warning_count\":0,\"instance_id\":\"4522\",\"leader\":true}}" 2020-04-10T10:06:13.891 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: } 2020-04-10T10:06:13.891 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: } 2020-04-10T10:06:13.891 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: } 2020-04-10T10:06:13.891 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:} 2020-04-10T10:06:13.901 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:13.901 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster2 mirror images 2020-04-10T10:06:13.901 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:13.901 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2 mirror images 2020-04-10T10:06:13.902 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror --namespace ls -l 2020-04-10T10:06:13.992 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:NAME SIZE PARENT FMT PROT LOCK 2020-04-10T10:06:13.993 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test 128 MiB 2 2020-04-10T10:06:13.993 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test1 128 MiB 2 2020-04-10T10:06:13.993 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test2 128 MiB 2 2020-04-10T10:06:13.993 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test2@snap1 128 MiB 2 2020-04-10T10:06:13.993 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test2@snap2 128 MiB 2 2020-04-10T10:06:13.993 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test2@snap_rename_20 128 MiB 2 2020-04-10T10:06:13.993 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test4 128 MiB 2 2020-04-10T10:06:13.994 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test4@snap1 128 MiB 2 yes 2020-04-10T10:06:13.994 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test4@snap2 128 MiB 2 yes 2020-04-10T10:06:13.994 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test_clone 128 MiB mirror_parent/test_parent@snap 2 2020-04-10T10:06:13.994 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test_clone1 128 MiB mirror_parent/test_parent@snap 2 excl 2020-04-10T10:06:13.994 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test_clone_v1 128 MiB mirror_parent/test_parent@snap 2 2020-04-10T10:06:13.994 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test_data_pool 128 MiB 2 2020-04-10T10:06:13.994 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test_data_pool@snap1 128 MiB 2 2020-04-10T10:06:13.995 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test_data_pool@snap2 128 MiB 2 2020-04-10T10:06:13.995 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test_force_promote 128 MiB 2 2020-04-10T10:06:13.995 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:13.995 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2 mirror mirror pool info 2020-04-10T10:06:13.996 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror --namespace mirror pool info 2020-04-10T10:06:13.996 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:13.996 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster2 mirror mirror pool info 2020-04-10T10:06:14.030 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Mode: image 2020-04-10T10:06:14.030 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Site Name: cluster2-DNE 2020-04-10T10:06:14.030 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:14.031 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Peer Sites: 2020-04-10T10:06:14.031 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:14.031 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:UUID: 90b6cfa2-5481-4edf-b462-5494aab62f6f 2020-04-10T10:06:14.031 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Name: cluster1-DNE 2020-04-10T10:06:14.031 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Mirror UUID: 620ddb00-0f49-4758-a216-8d9cc890d0fa 2020-04-10T10:06:14.031 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Direction: rx-tx 2020-04-10T10:06:14.031 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Client: client.mirror 2020-04-10T10:06:14.032 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:14.032 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2 mirror mirror pool status 2020-04-10T10:06:14.032 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster2 -p mirror --namespace mirror pool status --verbose 2020-04-10T10:06:14.033 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:14.033 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster2 mirror mirror pool status 2020-04-10T10:06:14.282 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:health: ERROR 2020-04-10T10:06:14.282 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:daemon health: OK 2020-04-10T10:06:14.283 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image health: ERROR 2020-04-10T10:06:14.283 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:images: 9 total 2020-04-10T10:06:14.283 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 8 unknown 2020-04-10T10:06:14.283 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 1 error 2020-04-10T10:06:14.284 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:14.284 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:DAEMONS 2020-04-10T10:06:14.284 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:service 4491: 2020-04-10T10:06:14.284 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: instance_id: 4502 2020-04-10T10:06:14.284 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: client_id: mirror.2 2020-04-10T10:06:14.284 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: hostname: smithi204 2020-04-10T10:06:14.285 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: version: 15.2.0-148-g39357d2 2020-04-10T10:06:14.285 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: leader: false 2020-04-10T10:06:14.285 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: health: OK 2020-04-10T10:06:14.285 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:14.286 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:service 4499: 2020-04-10T10:06:14.286 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: instance_id: 4510 2020-04-10T10:06:14.286 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: client_id: mirror.3 2020-04-10T10:06:14.286 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: hostname: smithi204 2020-04-10T10:06:14.286 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: version: 15.2.0-148-g39357d2 2020-04-10T10:06:14.287 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: leader: true 2020-04-10T10:06:14.287 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: health: OK 2020-04-10T10:06:14.287 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:14.287 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:14.332 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:IMAGES 2020-04-10T10:06:14.332 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test: 2020-04-10T10:06:14.332 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: global_id: a9af7d9e-1404-4730-9e18-c0418a915346 2020-04-10T10:06:14.332 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+stopped 2020-04-10T10:06:14.333 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: local image is primary 2020-04-10T10:06:14.333 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: service: mirror.3 on smithi204 2020-04-10T10:06:14.333 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 10:06:11 2020-04-10T10:06:14.333 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: peer_sites: 2020-04-10T10:06:14.334 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: name: cluster1-DNE 2020-04-10T10:06:14.334 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: down+stopped 2020-04-10T10:06:14.334 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: stopped 2020-04-10T10:06:14.334 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 07:11:16 2020-04-10T10:06:14.338 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:14.340 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test1: 2020-04-10T10:06:14.340 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: global_id: 56419286-13e9-49f6-a20d-2abe6ede083e 2020-04-10T10:06:14.340 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+stopped 2020-04-10T10:06:14.341 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: local image is primary 2020-04-10T10:06:14.341 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: service: mirror.3 on smithi204 2020-04-10T10:06:14.341 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 10:06:11 2020-04-10T10:06:14.341 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: peer_sites: 2020-04-10T10:06:14.341 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: name: cluster1-DNE 2020-04-10T10:06:14.342 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: down+stopped 2020-04-10T10:06:14.342 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: stopped 2020-04-10T10:06:14.342 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 07:11:14 2020-04-10T10:06:14.345 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:14.345 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test2: 2020-04-10T10:06:14.345 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: global_id: b583e294-8b69-4e57-9966-783595c454d6 2020-04-10T10:06:14.346 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+stopped 2020-04-10T10:06:14.346 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: local image is primary 2020-04-10T10:06:14.346 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: service: mirror.2 on smithi204 2020-04-10T10:06:14.346 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 07:12:31 2020-04-10T10:06:14.347 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: peer_sites: 2020-04-10T10:06:14.347 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: name: cluster1-DNE 2020-04-10T10:06:14.347 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: down+stopped 2020-04-10T10:06:14.348 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: stopped 2020-04-10T10:06:14.348 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 07:11:15 2020-04-10T10:06:14.355 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:14.356 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test_clone: 2020-04-10T10:06:14.356 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: global_id: 003078e2-ed6c-4f36-8bc5-ce0c44ef25d5 2020-04-10T10:06:14.356 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+stopped 2020-04-10T10:06:14.356 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: local image is primary 2020-04-10T10:06:14.356 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: service: mirror.3 on smithi204 2020-04-10T10:06:14.357 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 10:06:11 2020-04-10T10:06:14.357 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: peer_sites: 2020-04-10T10:06:14.357 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: name: cluster1-DNE 2020-04-10T10:06:14.357 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: down+stopped 2020-04-10T10:06:14.357 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: stopped 2020-04-10T10:06:14.358 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 07:11:14 2020-04-10T10:06:14.361 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:14.362 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test_clone1: 2020-04-10T10:06:14.362 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: global_id: 4b1961cb-5406-443f-90f1-3627ac4ffffd 2020-04-10T10:06:14.362 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+replaying 2020-04-10T10:06:14.362 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: replaying, {"replay_state": "idle", "remote_snapshot_timestamp": 1586502598, "local_snapshot_timestamp": 1586502598} 2020-04-10T10:06:14.363 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: service: mirror.2 on smithi204 2020-04-10T10:06:14.363 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 07:12:36 2020-04-10T10:06:14.363 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: peer_sites: 2020-04-10T10:06:14.363 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: name: cluster1-DNE 2020-04-10T10:06:14.363 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: down+stopped 2020-04-10T10:06:14.364 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: local image is primary 2020-04-10T10:06:14.364 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 07:11:15 2020-04-10T10:06:14.364 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:14.364 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test_clone_v1: 2020-04-10T10:06:14.365 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: global_id: 70d86755-f73a-4bbe-9493-1b79a191cc0c 2020-04-10T10:06:14.365 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+stopped 2020-04-10T10:06:14.365 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: local image is primary 2020-04-10T10:06:14.365 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: service: mirror.2 on smithi204 2020-04-10T10:06:14.365 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 07:12:31 2020-04-10T10:06:14.366 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: peer_sites: 2020-04-10T10:06:14.366 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: name: cluster1-DNE 2020-04-10T10:06:14.366 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: down+stopped 2020-04-10T10:06:14.366 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: stopped 2020-04-10T10:06:14.367 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 07:11:16 2020-04-10T10:06:14.367 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:14.367 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test_data_pool: 2020-04-10T10:06:14.367 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: global_id: a8183526-4331-4f0d-af4a-7c93c501e927 2020-04-10T10:06:14.367 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+stopped 2020-04-10T10:06:14.368 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: local image is primary 2020-04-10T10:06:14.368 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: service: mirror.3 on smithi204 2020-04-10T10:06:14.368 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 10:06:11 2020-04-10T10:06:14.368 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: peer_sites: 2020-04-10T10:06:14.369 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: name: cluster1-DNE 2020-04-10T10:06:14.369 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: down+stopped 2020-04-10T10:06:14.369 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: stopped 2020-04-10T10:06:14.369 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 07:11:14 2020-04-10T10:06:14.369 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:14.370 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test_force_promote: 2020-04-10T10:06:14.370 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: global_id: b192e243-6abc-405a-b798-b60b9d74691e 2020-04-10T10:06:14.371 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+stopped 2020-04-10T10:06:14.371 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: local image is primary 2020-04-10T10:06:14.371 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: service: mirror.2 on smithi204 2020-04-10T10:06:14.371 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 07:12:31 2020-04-10T10:06:14.372 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: peer_sites: 2020-04-10T10:06:14.372 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: name: cluster1-DNE 2020-04-10T10:06:14.372 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: down+stopped 2020-04-10T10:06:14.372 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: local image is primary 2020-04-10T10:06:14.373 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 07:11:16 2020-04-10T10:06:14.374 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:14.374 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror --namespace ls 2020-04-10T10:06:14.374 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:14.395 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test info 2020-04-10T10:06:14.396 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror --namespace info test 2020-04-10T10:06:14.396 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test info 2020-04-10T10:06:14.454 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:rbd image 'test': 2020-04-10T10:06:14.455 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: size 128 MiB in 32 objects 2020-04-10T10:06:14.455 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: order 22 (4 MiB objects) 2020-04-10T10:06:14.455 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: snapshot_count: 11 2020-04-10T10:06:14.455 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: id: 10e54538c85b 2020-04-10T10:06:14.455 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: block_name_prefix: rbd_data.10e54538c85b 2020-04-10T10:06:14.456 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: format: 2 2020-04-10T10:06:14.456 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: features: layering, exclusive-lock, journaling 2020-04-10T10:06:14.456 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: op_features: 2020-04-10T10:06:14.456 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: flags: 2020-04-10T10:06:14.457 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: create_timestamp: Fri Apr 10 07:06:26 2020 2020-04-10T10:06:14.457 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: access_timestamp: Fri Apr 10 07:08:21 2020 2020-04-10T10:06:14.457 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: modify_timestamp: Fri Apr 10 07:08:16 2020 2020-04-10T10:06:14.457 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: journal: 10e54538c85b 2020-04-10T10:06:14.458 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring state: enabled 2020-04-10T10:06:14.458 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring mode: snapshot 2020-04-10T10:06:14.458 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring global id: a9af7d9e-1404-4730-9e18-c0418a915346 2020-04-10T10:06:14.458 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring primary: true 2020-04-10T10:06:14.459 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:14.460 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test journal status 2020-04-10T10:06:14.461 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror --namespace journal status --image test 2020-04-10T10:06:14.462 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:14.463 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test journal status 2020-04-10T10:06:14.530 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:minimum_set: 0 2020-04-10T10:06:14.530 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:active_set: 0 2020-04-10T10:06:14.530 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:registered clients: 2020-04-10T10:06:14.531 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: [id=, commit_position=[positions=[[object_number=1, tag_tid=32, entry_tid=1], [object_number=0, tag_tid=32, entry_tid=0], [object_number=3, tag_tid=30, entry_tid=99], [object_number=2, tag_tid=30, entry_tid=98]]], state=connected] 2020-04-10T10:06:14.533 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:14.533 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test1 info 2020-04-10T10:06:14.533 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror --namespace info test1 2020-04-10T10:06:14.533 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:14.534 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test1 info 2020-04-10T10:06:14.595 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:rbd image 'test1': 2020-04-10T10:06:14.595 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: size 128 MiB in 32 objects 2020-04-10T10:06:14.595 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: order 22 (4 MiB objects) 2020-04-10T10:06:14.595 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: snapshot_count: 1 2020-04-10T10:06:14.596 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: id: 111968743338 2020-04-10T10:06:14.596 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: block_name_prefix: rbd_data.111968743338 2020-04-10T10:06:14.596 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: format: 2 2020-04-10T10:06:14.596 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: features: layering, exclusive-lock, journaling 2020-04-10T10:06:14.596 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: op_features: 2020-04-10T10:06:14.596 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: flags: 2020-04-10T10:06:14.596 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: create_timestamp: Fri Apr 10 07:06:38 2020 2020-04-10T10:06:14.597 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: access_timestamp: Fri Apr 10 07:06:38 2020 2020-04-10T10:06:14.597 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: modify_timestamp: Fri Apr 10 07:06:38 2020 2020-04-10T10:06:14.597 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: journal: 111968743338 2020-04-10T10:06:14.597 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring state: enabled 2020-04-10T10:06:14.597 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring mode: snapshot 2020-04-10T10:06:14.597 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring global id: 56419286-13e9-49f6-a20d-2abe6ede083e 2020-04-10T10:06:14.597 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring primary: true 2020-04-10T10:06:14.600 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:14.601 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test1 journal status 2020-04-10T10:06:14.601 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror --namespace journal status --image test1 2020-04-10T10:06:14.601 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:14.601 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test1 journal status 2020-04-10T10:06:14.666 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:minimum_set: 0 2020-04-10T10:06:14.666 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:active_set: 0 2020-04-10T10:06:14.666 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:registered clients: 2020-04-10T10:06:14.666 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: [id=, commit_position=[positions=[[object_number=1, tag_tid=4, entry_tid=1], [object_number=0, tag_tid=4, entry_tid=0], [object_number=3, tag_tid=2, entry_tid=99], [object_number=2, tag_tid=2, entry_tid=98]]], state=connected] 2020-04-10T10:06:14.670 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:14.670 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test2 info 2020-04-10T10:06:14.670 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:14.670 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test2 info 2020-04-10T10:06:14.670 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror --namespace info test2 2020-04-10T10:06:14.728 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:rbd image 'test2': 2020-04-10T10:06:14.728 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: size 128 MiB in 32 objects 2020-04-10T10:06:14.729 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: order 22 (4 MiB objects) 2020-04-10T10:06:14.729 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: snapshot_count: 4 2020-04-10T10:06:14.729 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: id: 12eff4a76192 2020-04-10T10:06:14.729 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: block_name_prefix: rbd_data.12eff4a76192 2020-04-10T10:06:14.729 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: format: 2 2020-04-10T10:06:14.729 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: features: layering, exclusive-lock, journaling 2020-04-10T10:06:14.729 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: op_features: 2020-04-10T10:06:14.730 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: flags: 2020-04-10T10:06:14.730 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: create_timestamp: Fri Apr 10 07:10:32 2020 2020-04-10T10:06:14.730 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: access_timestamp: Fri Apr 10 07:10:32 2020 2020-04-10T10:06:14.731 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: modify_timestamp: Fri Apr 10 07:10:32 2020 2020-04-10T10:06:14.731 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: journal: 12eff4a76192 2020-04-10T10:06:14.731 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring state: enabled 2020-04-10T10:06:14.731 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring mode: snapshot 2020-04-10T10:06:14.731 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring global id: b583e294-8b69-4e57-9966-783595c454d6 2020-04-10T10:06:14.731 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring primary: true 2020-04-10T10:06:14.732 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:14.732 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test2 journal status 2020-04-10T10:06:14.732 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:14.732 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test2 journal status 2020-04-10T10:06:14.733 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror --namespace journal status --image test2 2020-04-10T10:06:14.799 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:minimum_set: 0 2020-04-10T10:06:14.799 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:active_set: 0 2020-04-10T10:06:14.799 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:registered clients: 2020-04-10T10:06:14.799 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: [id=, commit_position=[positions=[[object_number=1, tag_tid=31, entry_tid=1], [object_number=0, tag_tid=31, entry_tid=0], [object_number=3, tag_tid=7, entry_tid=3], [object_number=2, tag_tid=7, entry_tid=2]]], state=connected] 2020-04-10T10:06:14.801 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:14.802 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test4 info 2020-04-10T10:06:14.802 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror --namespace info test4 2020-04-10T10:06:14.802 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:14.802 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test4 info 2020-04-10T10:06:14.857 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:rbd image 'test4': 2020-04-10T10:06:14.857 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: size 128 MiB in 32 objects 2020-04-10T10:06:14.857 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: order 22 (4 MiB objects) 2020-04-10T10:06:14.858 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: snapshot_count: 2 2020-04-10T10:06:14.858 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: id: 1328aa917c4d 2020-04-10T10:06:14.858 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: block_name_prefix: rbd_data.1328aa917c4d 2020-04-10T10:06:14.858 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: format: 2 2020-04-10T10:06:14.858 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: features: layering, exclusive-lock, journaling 2020-04-10T10:06:14.858 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: op_features: 2020-04-10T10:06:14.858 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: flags: 2020-04-10T10:06:14.859 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: create_timestamp: Fri Apr 10 07:10:44 2020 2020-04-10T10:06:14.859 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: access_timestamp: Fri Apr 10 07:10:44 2020 2020-04-10T10:06:14.859 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: modify_timestamp: Fri Apr 10 07:10:44 2020 2020-04-10T10:06:14.859 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: journal: 1328aa917c4d 2020-04-10T10:06:14.859 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring state: disabled 2020-04-10T10:06:14.861 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:14.861 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test4 journal status 2020-04-10T10:06:14.861 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:14.861 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test4 journal status 2020-04-10T10:06:14.862 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror --namespace journal status --image test4 2020-04-10T10:06:14.921 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:minimum_set: 0 2020-04-10T10:06:14.921 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:active_set: 0 2020-04-10T10:06:14.922 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:registered clients: 2020-04-10T10:06:14.922 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: [id=, commit_position=[positions=[[object_number=3, tag_tid=9, entry_tid=3], [object_number=2, tag_tid=9, entry_tid=2], [object_number=1, tag_tid=9, entry_tid=1], [object_number=0, tag_tid=9, entry_tid=0]]], state=connected] 2020-04-10T10:06:14.924 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:14.924 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test_clone info 2020-04-10T10:06:14.925 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror --namespace info test_clone 2020-04-10T10:06:14.925 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:14.925 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test_clone info 2020-04-10T10:06:14.996 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:rbd image 'test_clone': 2020-04-10T10:06:14.996 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: size 128 MiB in 32 objects 2020-04-10T10:06:14.996 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: order 22 (4 MiB objects) 2020-04-10T10:06:14.996 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: snapshot_count: 1 2020-04-10T10:06:14.997 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: id: 1268616d9c91 2020-04-10T10:06:14.997 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: block_name_prefix: rbd_data.1268616d9c91 2020-04-10T10:06:14.997 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: format: 2 2020-04-10T10:06:14.997 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: features: layering, exclusive-lock, journaling 2020-04-10T10:06:14.997 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: op_features: 2020-04-10T10:06:14.997 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: flags: 2020-04-10T10:06:14.997 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: create_timestamp: Fri Apr 10 07:09:44 2020 2020-04-10T10:06:14.998 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: access_timestamp: Fri Apr 10 07:09:44 2020 2020-04-10T10:06:14.998 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: modify_timestamp: Fri Apr 10 07:09:44 2020 2020-04-10T10:06:14.998 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: parent: mirror_parent/test_parent@snap 2020-04-10T10:06:14.998 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: overlap: 128 MiB 2020-04-10T10:06:14.999 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: journal: 1268616d9c91 2020-04-10T10:06:14.999 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring state: enabled 2020-04-10T10:06:14.999 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring mode: snapshot 2020-04-10T10:06:14.999 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring global id: 003078e2-ed6c-4f36-8bc5-ce0c44ef25d5 2020-04-10T10:06:14.999 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring primary: true 2020-04-10T10:06:15.002 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:15.002 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test_clone journal status 2020-04-10T10:06:15.002 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror --namespace journal status --image test_clone 2020-04-10T10:06:15.002 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.002 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test_clone journal status 2020-04-10T10:06:15.085 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:minimum_set: 0 2020-04-10T10:06:15.086 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:active_set: 0 2020-04-10T10:06:15.086 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:registered clients: 2020-04-10T10:06:15.086 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: [id=, commit_position=[positions=[[object_number=1, tag_tid=4, entry_tid=1], [object_number=0, tag_tid=4, entry_tid=0], [object_number=3, tag_tid=1, entry_tid=99], [object_number=2, tag_tid=1, entry_tid=98]]], state=connected] 2020-04-10T10:06:15.089 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.089 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test_clone1 info 2020-04-10T10:06:15.089 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:15.089 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test_clone1 info 2020-04-10T10:06:15.089 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror --namespace info test_clone1 2020-04-10T10:06:15.173 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:rbd image 'test_clone1': 2020-04-10T10:06:15.173 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: size 128 MiB in 32 objects 2020-04-10T10:06:15.173 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: order 22 (4 MiB objects) 2020-04-10T10:06:15.173 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: snapshot_count: 1 2020-04-10T10:06:15.173 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: id: 1189d1969f13 2020-04-10T10:06:15.173 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: block_name_prefix: rbd_data.1189d1969f13 2020-04-10T10:06:15.174 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: format: 2 2020-04-10T10:06:15.174 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: features: layering, exclusive-lock, journaling, non-primary 2020-04-10T10:06:15.174 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: op_features: 2020-04-10T10:06:15.174 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: flags: 2020-04-10T10:06:15.174 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: create_timestamp: Fri Apr 10 07:10:00 2020 2020-04-10T10:06:15.174 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: access_timestamp: Fri Apr 10 07:10:00 2020 2020-04-10T10:06:15.174 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: modify_timestamp: Fri Apr 10 07:10:00 2020 2020-04-10T10:06:15.175 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: parent: mirror_parent/test_parent@snap 2020-04-10T10:06:15.175 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: overlap: 128 MiB 2020-04-10T10:06:15.175 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: journal: 1189d1969f13 2020-04-10T10:06:15.175 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring state: enabled 2020-04-10T10:06:15.175 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring mode: snapshot 2020-04-10T10:06:15.176 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring global id: 4b1961cb-5406-443f-90f1-3627ac4ffffd 2020-04-10T10:06:15.176 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring primary: false 2020-04-10T10:06:15.178 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.178 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test_clone1 journal status 2020-04-10T10:06:15.178 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:15.178 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test_clone1 journal status 2020-04-10T10:06:15.178 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror --namespace journal status --image test_clone1 2020-04-10T10:06:15.257 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:minimum_set: 0 2020-04-10T10:06:15.258 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:active_set: 0 2020-04-10T10:06:15.258 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:registered clients: 2020-04-10T10:06:15.258 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: [id=, commit_position=[positions=[]], state=connected] 2020-04-10T10:06:15.261 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:15.262 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test_clone_v1 info 2020-04-10T10:06:15.262 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror --namespace info test_clone_v1 2020-04-10T10:06:15.262 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.262 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test_clone_v1 info 2020-04-10T10:06:15.336 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:rbd image 'test_clone_v1': 2020-04-10T10:06:15.338 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: size 128 MiB in 32 objects 2020-04-10T10:06:15.338 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: order 22 (4 MiB objects) 2020-04-10T10:06:15.338 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: snapshot_count: 1 2020-04-10T10:06:15.339 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: id: 1287c927b049 2020-04-10T10:06:15.344 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: block_name_prefix: rbd_data.1287c927b049 2020-04-10T10:06:15.345 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: format: 2 2020-04-10T10:06:15.345 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: features: layering, exclusive-lock, journaling 2020-04-10T10:06:15.345 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: op_features: 2020-04-10T10:06:15.345 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: flags: 2020-04-10T10:06:15.346 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: create_timestamp: Fri Apr 10 07:09:58 2020 2020-04-10T10:06:15.346 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: access_timestamp: Fri Apr 10 07:09:58 2020 2020-04-10T10:06:15.346 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: modify_timestamp: Fri Apr 10 07:09:58 2020 2020-04-10T10:06:15.347 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: parent: mirror_parent/test_parent@snap 2020-04-10T10:06:15.347 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: overlap: 128 MiB 2020-04-10T10:06:15.347 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: journal: 1287c927b049 2020-04-10T10:06:15.348 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring state: enabled 2020-04-10T10:06:15.348 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring mode: snapshot 2020-04-10T10:06:15.348 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring global id: 70d86755-f73a-4bbe-9493-1b79a191cc0c 2020-04-10T10:06:15.348 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring primary: true 2020-04-10T10:06:15.348 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.348 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test_clone_v1 journal status 2020-04-10T10:06:15.351 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:15.351 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test_clone_v1 journal status 2020-04-10T10:06:15.352 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror --namespace journal status --image test_clone_v1 2020-04-10T10:06:15.424 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:minimum_set: 0 2020-04-10T10:06:15.424 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:active_set: 0 2020-04-10T10:06:15.425 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:registered clients: 2020-04-10T10:06:15.425 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: [id=, commit_position=[positions=[[object_number=1, tag_tid=1, entry_tid=1], [object_number=0, tag_tid=1, entry_tid=0]]], state=connected] 2020-04-10T10:06:15.427 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.428 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test_data_pool info 2020-04-10T10:06:15.428 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:15.428 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test_data_pool info 2020-04-10T10:06:15.428 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror --namespace info test_data_pool 2020-04-10T10:06:15.482 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:rbd image 'test_data_pool': 2020-04-10T10:06:15.483 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: size 128 MiB in 32 objects 2020-04-10T10:06:15.483 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: order 22 (4 MiB objects) 2020-04-10T10:06:15.483 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: snapshot_count: 3 2020-04-10T10:06:15.483 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: id: 12c17d32928c 2020-04-10T10:06:15.483 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: data_pool: mirror_parent 2020-04-10T10:06:15.483 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: block_name_prefix: rbd_data.3.12c17d32928c 2020-04-10T10:06:15.483 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: format: 2 2020-04-10T10:06:15.484 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: features: layering, exclusive-lock, journaling, data-pool 2020-04-10T10:06:15.484 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: op_features: 2020-04-10T10:06:15.484 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: flags: 2020-04-10T10:06:15.484 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: create_timestamp: Fri Apr 10 07:10:20 2020 2020-04-10T10:06:15.484 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: access_timestamp: Fri Apr 10 07:10:20 2020 2020-04-10T10:06:15.484 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: modify_timestamp: Fri Apr 10 07:10:20 2020 2020-04-10T10:06:15.485 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: journal: 12c17d32928c 2020-04-10T10:06:15.485 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring state: enabled 2020-04-10T10:06:15.485 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring mode: snapshot 2020-04-10T10:06:15.485 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring global id: a8183526-4331-4f0d-af4a-7c93c501e927 2020-04-10T10:06:15.485 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring primary: true 2020-04-10T10:06:15.486 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:15.486 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test_data_pool journal status 2020-04-10T10:06:15.487 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror --namespace journal status --image test_data_pool 2020-04-10T10:06:15.487 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.487 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test_data_pool journal status 2020-04-10T10:06:15.549 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:minimum_set: 0 2020-04-10T10:06:15.550 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:active_set: 0 2020-04-10T10:06:15.550 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:registered clients: 2020-04-10T10:06:15.550 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: [id=, commit_position=[positions=[[object_number=1, tag_tid=7, entry_tid=1], [object_number=0, tag_tid=7, entry_tid=0], [object_number=3, tag_tid=5, entry_tid=99], [object_number=2, tag_tid=5, entry_tid=98]]], state=connected] 2020-04-10T10:06:15.553 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:15.553 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test_force_promote info 2020-04-10T10:06:15.553 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror --namespace info test_force_promote 2020-04-10T10:06:15.553 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.554 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test_force_promote info 2020-04-10T10:06:15.616 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:rbd image 'test_force_promote': 2020-04-10T10:06:15.617 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: size 128 MiB in 32 objects 2020-04-10T10:06:15.617 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: order 22 (4 MiB objects) 2020-04-10T10:06:15.617 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: snapshot_count: 1 2020-04-10T10:06:15.617 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: id: 12381bb6482e 2020-04-10T10:06:15.617 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: block_name_prefix: rbd_data.12381bb6482e 2020-04-10T10:06:15.617 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: format: 2 2020-04-10T10:06:15.617 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: features: layering, exclusive-lock, journaling 2020-04-10T10:06:15.618 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: op_features: 2020-04-10T10:06:15.618 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: flags: 2020-04-10T10:06:15.618 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: create_timestamp: Fri Apr 10 07:09:20 2020 2020-04-10T10:06:15.618 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: access_timestamp: Fri Apr 10 07:09:20 2020 2020-04-10T10:06:15.618 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: modify_timestamp: Fri Apr 10 07:09:20 2020 2020-04-10T10:06:15.618 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: journal: 12381bb6482e 2020-04-10T10:06:15.619 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring state: enabled 2020-04-10T10:06:15.619 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring mode: snapshot 2020-04-10T10:06:15.619 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring global id: b192e243-6abc-405a-b798-b60b9d74691e 2020-04-10T10:06:15.619 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring primary: true 2020-04-10T10:06:15.621 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.621 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test_force_promote journal status 2020-04-10T10:06:15.621 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:15.621 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test_force_promote journal status 2020-04-10T10:06:15.621 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror --namespace journal status --image test_force_promote 2020-04-10T10:06:15.691 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:minimum_set: 0 2020-04-10T10:06:15.691 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:active_set: 0 2020-04-10T10:06:15.691 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:registered clients: 2020-04-10T10:06:15.692 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: [id=, commit_position=[positions=[[object_number=1, tag_tid=5, entry_tid=101], [object_number=0, tag_tid=5, entry_tid=100], [object_number=3, tag_tid=5, entry_tid=99], [object_number=2, tag_tid=5, entry_tid=98]]], state=connected] 2020-04-10T10:06:15.694 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:15.694 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2 mirror rbd_mirroring omap vals 2020-04-10T10:06:15.695 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rados --cluster cluster2 -p mirror --namespace listomapvals rbd_mirroring 2020-04-10T10:06:15.695 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.695 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster2 mirror rbd_mirroring omap vals 2020-04-10T10:06:15.741 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:global_003078e2-ed6c-4f36-8bc5-ce0c44ef25d5 2020-04-10T10:06:15.741 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (16 bytes) : 2020-04-10T10:06:15.741 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 0c 00 00 00 31 32 36 38 36 31 36 64 39 63 39 31 |....1268616d9c91| 2020-04-10T10:06:15.742 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 2020-04-10T10:06:15.743 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.743 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:global_4b1961cb-5406-443f-90f1-3627ac4ffffd 2020-04-10T10:06:15.744 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (16 bytes) : 2020-04-10T10:06:15.744 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 0c 00 00 00 31 31 38 39 64 31 39 36 39 66 31 33 |....1189d1969f13| 2020-04-10T10:06:15.744 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 2020-04-10T10:06:15.744 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.744 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:global_56419286-13e9-49f6-a20d-2abe6ede083e 2020-04-10T10:06:15.744 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (16 bytes) : 2020-04-10T10:06:15.745 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 0c 00 00 00 31 31 31 39 36 38 37 34 33 33 33 38 |....111968743338| 2020-04-10T10:06:15.745 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 2020-04-10T10:06:15.745 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.745 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:global_70d86755-f73a-4bbe-9493-1b79a191cc0c 2020-04-10T10:06:15.745 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (16 bytes) : 2020-04-10T10:06:15.745 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 0c 00 00 00 31 32 38 37 63 39 32 37 62 30 34 39 |....1287c927b049| 2020-04-10T10:06:15.745 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 2020-04-10T10:06:15.746 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.746 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:global_79225ce2-9203-4752-bc9b-35a11186acdb 2020-04-10T10:06:15.746 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (16 bytes) : 2020-04-10T10:06:15.746 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 0c 00 00 00 31 31 39 65 63 30 31 64 32 38 34 66 |....119ec01d284f| 2020-04-10T10:06:15.746 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 2020-04-10T10:06:15.746 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.746 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:global_a8183526-4331-4f0d-af4a-7c93c501e927 2020-04-10T10:06:15.747 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (16 bytes) : 2020-04-10T10:06:15.747 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 0c 00 00 00 31 32 63 31 37 64 33 32 39 32 38 63 |....12c17d32928c| 2020-04-10T10:06:15.747 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 2020-04-10T10:06:15.747 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.748 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:global_a9af7d9e-1404-4730-9e18-c0418a915346 2020-04-10T10:06:15.748 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (16 bytes) : 2020-04-10T10:06:15.750 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 0c 00 00 00 31 30 65 35 34 35 33 38 63 38 35 62 |....10e54538c85b| 2020-04-10T10:06:15.750 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 2020-04-10T10:06:15.750 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.751 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:global_b192e243-6abc-405a-b798-b60b9d74691e 2020-04-10T10:06:15.751 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (16 bytes) : 2020-04-10T10:06:15.751 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 0c 00 00 00 31 32 33 38 31 62 62 36 34 38 32 65 |....12381bb6482e| 2020-04-10T10:06:15.751 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 2020-04-10T10:06:15.751 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.751 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:global_b583e294-8b69-4e57-9966-783595c454d6 2020-04-10T10:06:15.751 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (16 bytes) : 2020-04-10T10:06:15.752 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 0c 00 00 00 31 32 65 66 66 34 61 37 36 31 39 32 |....12eff4a76192| 2020-04-10T10:06:15.752 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 2020-04-10T10:06:15.752 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.752 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image_10e54538c85b 2020-04-10T10:06:15.752 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (48 bytes) : 2020-04-10T10:06:15.752 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 02 01 2a 00 00 00 24 00 00 00 61 39 61 66 37 64 |..*...$...a9af7d| 2020-04-10T10:06:15.752 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 39 65 2d 31 34 30 34 2d 34 37 33 30 2d 39 65 31 |9e-1404-4730-9e1| 2020-04-10T10:06:15.753 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 38 2d 63 30 34 31 38 61 39 31 35 33 34 36 01 01 |8-c0418a915346..| 2020-04-10T10:06:15.753 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 2020-04-10T10:06:15.754 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.754 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image_111968743338 2020-04-10T10:06:15.754 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (48 bytes) : 2020-04-10T10:06:15.755 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 02 01 2a 00 00 00 24 00 00 00 35 36 34 31 39 32 |..*...$...564192| 2020-04-10T10:06:15.755 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 38 36 2d 31 33 65 39 2d 34 39 66 36 2d 61 32 30 |86-13e9-49f6-a20| 2020-04-10T10:06:15.755 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 64 2d 32 61 62 65 36 65 64 65 30 38 33 65 01 01 |d-2abe6ede083e..| 2020-04-10T10:06:15.756 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 2020-04-10T10:06:15.756 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.756 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image_1189d1969f13 2020-04-10T10:06:15.756 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (48 bytes) : 2020-04-10T10:06:15.757 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 02 01 2a 00 00 00 24 00 00 00 34 62 31 39 36 31 |..*...$...4b1961| 2020-04-10T10:06:15.757 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 63 62 2d 35 34 30 36 2d 34 34 33 66 2d 39 30 66 |cb-5406-443f-90f| 2020-04-10T10:06:15.757 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 31 2d 33 36 32 37 61 63 34 66 66 66 66 64 01 01 |1-3627ac4ffffd..| 2020-04-10T10:06:15.757 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 2020-04-10T10:06:15.757 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.757 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image_119ec01d284f 2020-04-10T10:06:15.757 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (48 bytes) : 2020-04-10T10:06:15.758 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 02 01 2a 00 00 00 24 00 00 00 37 39 32 32 35 63 |..*...$...79225c| 2020-04-10T10:06:15.758 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 65 32 2d 39 32 30 33 2d 34 37 35 32 2d 62 63 39 |e2-9203-4752-bc9| 2020-04-10T10:06:15.758 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 62 2d 33 35 61 31 31 31 38 36 61 63 64 62 03 01 |b-35a11186acdb..| 2020-04-10T10:06:15.758 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 2020-04-10T10:06:15.759 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.759 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image_12381bb6482e 2020-04-10T10:06:15.759 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (48 bytes) : 2020-04-10T10:06:15.760 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 02 01 2a 00 00 00 24 00 00 00 62 31 39 32 65 32 |..*...$...b192e2| 2020-04-10T10:06:15.760 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 34 33 2d 36 61 62 63 2d 34 30 35 61 2d 62 37 39 |43-6abc-405a-b79| 2020-04-10T10:06:15.761 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 38 2d 62 36 30 62 39 64 37 34 36 39 31 65 01 01 |8-b60b9d74691e..| 2020-04-10T10:06:15.762 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 2020-04-10T10:06:15.762 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.762 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image_1268616d9c91 2020-04-10T10:06:15.763 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (48 bytes) : 2020-04-10T10:06:15.763 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 02 01 2a 00 00 00 24 00 00 00 30 30 33 30 37 38 |..*...$...003078| 2020-04-10T10:06:15.764 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 65 32 2d 65 64 36 63 2d 34 66 33 36 2d 38 62 63 |e2-ed6c-4f36-8bc| 2020-04-10T10:06:15.764 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 35 2d 63 65 30 63 34 34 65 66 32 35 64 35 01 01 |5-ce0c44ef25d5..| 2020-04-10T10:06:15.764 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 2020-04-10T10:06:15.765 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.765 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image_1287c927b049 2020-04-10T10:06:15.765 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (48 bytes) : 2020-04-10T10:06:15.765 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 02 01 2a 00 00 00 24 00 00 00 37 30 64 38 36 37 |..*...$...70d867| 2020-04-10T10:06:15.765 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 35 35 2d 66 37 33 61 2d 34 62 62 65 2d 39 34 39 |55-f73a-4bbe-949| 2020-04-10T10:06:15.765 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 33 2d 31 62 37 39 61 31 39 31 63 63 30 63 01 01 |3-1b79a191cc0c..| 2020-04-10T10:06:15.765 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 2020-04-10T10:06:15.766 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.766 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image_12c17d32928c 2020-04-10T10:06:15.766 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (48 bytes) : 2020-04-10T10:06:15.767 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 02 01 2a 00 00 00 24 00 00 00 61 38 31 38 33 35 |..*...$...a81835| 2020-04-10T10:06:15.767 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 32 36 2d 34 33 33 31 2d 34 66 30 64 2d 61 66 34 |26-4331-4f0d-af4| 2020-04-10T10:06:15.768 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 61 2d 37 63 39 33 63 35 30 31 65 39 32 37 01 01 |a-7c93c501e927..| 2020-04-10T10:06:15.768 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 2020-04-10T10:06:15.768 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.768 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image_12eff4a76192 2020-04-10T10:06:15.769 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (48 bytes) : 2020-04-10T10:06:15.769 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 02 01 2a 00 00 00 24 00 00 00 62 35 38 33 65 32 |..*...$...b583e2| 2020-04-10T10:06:15.769 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 39 34 2d 38 62 36 39 2d 34 65 35 37 2d 39 39 36 |94-8b69-4e57-996| 2020-04-10T10:06:15.769 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 36 2d 37 38 33 35 39 35 63 34 35 34 64 36 01 01 |6-783595c454d6..| 2020-04-10T10:06:15.769 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 2020-04-10T10:06:15.769 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.769 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:mirror_mode 2020-04-10T10:06:15.771 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (4 bytes) : 2020-04-10T10:06:15.771 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 00 00 00 |....| 2020-04-10T10:06:15.771 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000004 2020-04-10T10:06:15.771 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.771 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:mirror_peer_90b6cfa2-5481-4edf-b462-5494aab62f6f 2020-04-10T10:06:15.771 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (136 bytes) : 2020-04-10T10:06:15.772 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 02 01 82 00 00 00 24 00 00 00 39 30 62 36 63 66 |......$...90b6cf| 2020-04-10T10:06:15.772 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 61 32 2d 35 34 38 31 2d 34 65 64 66 2d 62 34 36 |a2-5481-4edf-b46| 2020-04-10T10:06:15.772 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 32 2d 35 34 39 34 61 61 62 36 32 66 36 66 0c 00 |2-5494aab62f6f..| 2020-04-10T10:06:15.772 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 63 6c 75 73 74 65 72 31 2d 44 4e 45 0d 00 |..cluster1-DNE..| 2020-04-10T10:06:15.772 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 00 00 63 6c 69 65 6e 74 2e 6d 69 72 72 6f 72 ff |..client.mirror.| 2020-04-10T10:06:15.772 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 ff ff ff ff ff ff ff 02 24 00 00 00 36 32 30 64 |........$...620d| 2020-04-10T10:06:15.772 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000060 64 62 30 30 2d 30 66 34 39 2d 34 37 35 38 2d 61 |db00-0f49-4758-a| 2020-04-10T10:06:15.773 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000070 32 31 36 2d 38 64 39 63 63 38 39 30 64 30 66 61 |216-8d9cc890d0fa| 2020-04-10T10:06:15.773 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000080 0d 1c 90 5e 25 bc c2 1c |...^%...| 2020-04-10T10:06:15.773 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000088 2020-04-10T10:06:15.773 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.773 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:mirror_uuid 2020-04-10T10:06:15.774 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (36 bytes) : 2020-04-10T10:06:15.774 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 38 32 35 33 66 64 65 32 2d 38 36 66 38 2d 34 34 |8253fde2-86f8-44| 2020-04-10T10:06:15.774 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 33 61 2d 62 34 62 66 2d 65 63 37 64 61 65 32 66 |3a-b4bf-ec7dae2f| 2020-04-10T10:06:15.774 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 61 66 61 33 |afa3| 2020-04-10T10:06:15.774 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000024 2020-04-10T10:06:15.774 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.775 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:remote_status_global_003078e2-ed6c-4f36-8bc5-ce0c44ef25d5_620ddb00-0f49-4758-a216-8d9cc890d0fa 2020-04-10T10:06:15.775 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (77 bytes) : 2020-04-10T10:06:15.775 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 24 11 00 00 00 00 00 00 01 |..,....$........| 2020-04-10T10:06:15.775 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 b6 31 b9 ca 10 00 |...........1....| 2020-04-10T10:06:15.775 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T10:06:15.776 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 15 00 00 00 06 07 00 00 00 73 74 6f |.............sto| 2020-04-10T10:06:15.776 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 70 70 65 64 12 1c 90 5e f2 97 d8 17 00 |pped...^.....| 2020-04-10T10:06:15.776 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000004d 2020-04-10T10:06:15.776 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.776 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:remote_status_global_3876de2e-b17d-4c86-94c4-90a11916f44e_620ddb00-0f49-4758-a216-8d9cc890d0fa 2020-04-10T10:06:15.777 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (96 bytes) : 2020-04-10T10:06:15.777 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 28 11 00 00 00 00 00 00 01 |..,....(........| 2020-04-10T10:06:15.777 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 1b 92 fa 21 10 00 |.............!..| 2020-04-10T10:06:15.777 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T10:06:15.777 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 28 00 00 00 01 1a 00 00 00 65 72 72 |....(........err| 2020-04-10T10:06:15.777 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 6f 72 20 62 6f 6f 74 73 74 72 61 70 70 69 6e 67 |or bootstrapping| 2020-04-10T10:06:15.777 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 20 72 65 70 6c 61 79 14 1c 90 5e 71 cc 13 21 00 | replay...^q..!.| 2020-04-10T10:06:15.778 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000060 2020-04-10T10:06:15.778 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.778 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:remote_status_global_4334771a-4312-4fab-8e12-36d78acc7af1_620ddb00-0f49-4758-a216-8d9cc890d0fa 2020-04-10T10:06:15.778 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (96 bytes) : 2020-04-10T10:06:15.778 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 2a 11 00 00 00 00 00 00 01 |..,....*........| 2020-04-10T10:06:15.778 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 ed ce d6 f8 10 00 |................| 2020-04-10T10:06:15.779 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T10:06:15.779 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 28 00 00 00 01 1a 00 00 00 65 72 72 |....(........err| 2020-04-10T10:06:15.779 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 6f 72 20 62 6f 6f 74 73 74 72 61 70 70 69 6e 67 |or bootstrapping| 2020-04-10T10:06:15.779 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 20 72 65 70 6c 61 79 14 1c 90 5e fc 5d 37 21 00 | replay...^.]7!.| 2020-04-10T10:06:15.779 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000060 2020-04-10T10:06:15.779 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.779 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:remote_status_global_4b1961cb-5406-443f-90f1-3627ac4ffffd_620ddb00-0f49-4758-a216-8d9cc890d0fa 2020-04-10T10:06:15.780 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (92 bytes) : 2020-04-10T10:06:15.780 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 26 11 00 00 00 00 00 00 01 |..,....&........| 2020-04-10T10:06:15.780 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 d5 a9 fe 0b 10 00 |................| 2020-04-10T10:06:15.780 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T10:06:15.780 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 24 00 00 00 06 16 00 00 00 6c 6f 63 |....$........loc| 2020-04-10T10:06:15.780 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 61 6c 20 69 6d 61 67 65 20 69 73 20 70 72 69 6d |al image is prim| 2020-04-10T10:06:15.780 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 61 72 79 13 1c 90 5e 92 fa f6 1a 00 |ary...^.....| 2020-04-10T10:06:15.781 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000005c 2020-04-10T10:06:15.781 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.781 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:remote_status_global_56419286-13e9-49f6-a20d-2abe6ede083e_620ddb00-0f49-4758-a216-8d9cc890d0fa 2020-04-10T10:06:15.781 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (77 bytes) : 2020-04-10T10:06:15.781 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 24 11 00 00 00 00 00 00 01 |..,....$........| 2020-04-10T10:06:15.781 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 b6 31 b9 ca 10 00 |...........1....| 2020-04-10T10:06:15.781 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T10:06:15.782 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 15 00 00 00 06 07 00 00 00 73 74 6f |.............sto| 2020-04-10T10:06:15.782 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 70 70 65 64 12 1c 90 5e d3 cb 8d 17 00 |pped...^.....| 2020-04-10T10:06:15.782 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000004d 2020-04-10T10:06:15.782 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.782 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:remote_status_global_5d3e94e2-1c6c-4146-b461-67ecadf338c4_620ddb00-0f49-4758-a216-8d9cc890d0fa 2020-04-10T10:06:15.782 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (96 bytes) : 2020-04-10T10:06:15.782 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 2a 11 00 00 00 00 00 00 01 |..,....*........| 2020-04-10T10:06:15.783 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 ed ce d6 f8 10 00 |................| 2020-04-10T10:06:15.783 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T10:06:15.783 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 28 00 00 00 01 1a 00 00 00 65 72 72 |....(........err| 2020-04-10T10:06:15.783 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 6f 72 20 62 6f 6f 74 73 74 72 61 70 70 69 6e 67 |or bootstrapping| 2020-04-10T10:06:15.783 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 20 72 65 70 6c 61 79 14 1c 90 5e 7d be 5d 21 00 | replay...^}.]!.| 2020-04-10T10:06:15.783 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000060 2020-04-10T10:06:15.783 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.784 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:remote_status_global_687e9f7d-ae46-4324-b8f7-7c16605e0fa6_620ddb00-0f49-4758-a216-8d9cc890d0fa 2020-04-10T10:06:15.784 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (96 bytes) : 2020-04-10T10:06:15.784 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 26 11 00 00 00 00 00 00 01 |..,....&........| 2020-04-10T10:06:15.784 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 d5 a9 fe 0b 10 00 |................| 2020-04-10T10:06:15.784 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T10:06:15.785 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 28 00 00 00 01 1a 00 00 00 65 72 72 |....(........err| 2020-04-10T10:06:15.785 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 6f 72 20 62 6f 6f 74 73 74 72 61 70 70 69 6e 67 |or bootstrapping| 2020-04-10T10:06:15.785 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 20 72 65 70 6c 61 79 13 1c 90 5e 59 4a f6 19 00 | replay...^YJ...| 2020-04-10T10:06:15.785 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000060 2020-04-10T10:06:15.785 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.785 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:remote_status_global_697ea219-b252-4c33-ba20-9b31ea5be82a_620ddb00-0f49-4758-a216-8d9cc890d0fa 2020-04-10T10:06:15.785 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (96 bytes) : 2020-04-10T10:06:15.786 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 24 11 00 00 00 00 00 00 01 |..,....$........| 2020-04-10T10:06:15.786 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 b6 31 b9 ca 10 00 |...........1....| 2020-04-10T10:06:15.786 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T10:06:15.786 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 28 00 00 00 01 1a 00 00 00 65 72 72 |....(........err| 2020-04-10T10:06:15.786 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 6f 72 20 62 6f 6f 74 73 74 72 61 70 70 69 6e 67 |or bootstrapping| 2020-04-10T10:06:15.786 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 20 72 65 70 6c 61 79 0d 1c 90 5e c4 f6 57 16 00 | replay...^..W..| 2020-04-10T10:06:15.786 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000060 2020-04-10T10:06:15.787 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.787 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:remote_status_global_70d86755-f73a-4bbe-9493-1b79a191cc0c_620ddb00-0f49-4758-a216-8d9cc890d0fa 2020-04-10T10:06:15.787 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (77 bytes) : 2020-04-10T10:06:15.787 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 28 11 00 00 00 00 00 00 01 |..,....(........| 2020-04-10T10:06:15.787 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 1b 92 fa 21 10 00 |.............!..| 2020-04-10T10:06:15.787 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T10:06:15.787 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 15 00 00 00 06 07 00 00 00 73 74 6f |.............sto| 2020-04-10T10:06:15.788 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 70 70 65 64 14 1c 90 5e 65 0d de 26 00 |pped...^e..&.| 2020-04-10T10:06:15.788 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000004d 2020-04-10T10:06:15.788 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.788 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:remote_status_global_79225ce2-9203-4752-bc9b-35a11186acdb_620ddb00-0f49-4758-a216-8d9cc890d0fa 2020-04-10T10:06:15.788 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (97 bytes) : 2020-04-10T10:06:15.788 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 28 11 00 00 00 00 00 00 01 |..,....(........| 2020-04-10T10:06:15.788 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 1b 92 fa 21 10 00 |.............!..| 2020-04-10T10:06:15.789 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T10:06:15.789 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 29 00 00 00 00 1b 00 00 00 72 65 6d |....)........rem| 2020-04-10T10:06:15.789 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 6f 74 65 20 69 6d 61 67 65 20 69 73 20 6e 6f 6e |ote image is non| 2020-04-10T10:06:15.789 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 2d 70 72 69 6d 61 72 79 14 1c 90 5e 84 ba 8e 21 |-primary...^...!| 2020-04-10T10:06:15.789 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000060 00 |.| 2020-04-10T10:06:15.789 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000061 2020-04-10T10:06:15.789 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.790 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:remote_status_global_925c4d02-9086-4edd-b4f3-79c1b7e3734c_620ddb00-0f49-4758-a216-8d9cc890d0fa 2020-04-10T10:06:15.790 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (96 bytes) : 2020-04-10T10:06:15.790 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 26 11 00 00 00 00 00 00 01 |..,....&........| 2020-04-10T10:06:15.790 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 d5 a9 fe 0b 10 00 |................| 2020-04-10T10:06:15.790 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T10:06:15.790 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 28 00 00 00 01 1a 00 00 00 65 72 72 |....(........err| 2020-04-10T10:06:15.790 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 6f 72 20 62 6f 6f 74 73 74 72 61 70 70 69 6e 67 |or bootstrapping| 2020-04-10T10:06:15.790 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 20 72 65 70 6c 61 79 13 1c 90 5e e5 b1 11 1a 00 | replay...^.....| 2020-04-10T10:06:15.791 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000060 2020-04-10T10:06:15.791 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.791 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:remote_status_global_a8183526-4331-4f0d-af4a-7c93c501e927_620ddb00-0f49-4758-a216-8d9cc890d0fa 2020-04-10T10:06:15.791 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (77 bytes) : 2020-04-10T10:06:15.791 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 24 11 00 00 00 00 00 00 01 |..,....$........| 2020-04-10T10:06:15.791 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 b6 31 b9 ca 10 00 |...........1....| 2020-04-10T10:06:15.791 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T10:06:15.792 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 15 00 00 00 06 07 00 00 00 73 74 6f |.............sto| 2020-04-10T10:06:15.792 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 70 70 65 64 12 1c 90 5e 24 58 f6 17 00 |pped...^$X...| 2020-04-10T10:06:15.792 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000004d 2020-04-10T10:06:15.792 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.792 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:remote_status_global_a9af7d9e-1404-4730-9e18-c0418a915346_620ddb00-0f49-4758-a216-8d9cc890d0fa 2020-04-10T10:06:15.792 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (77 bytes) : 2020-04-10T10:06:15.792 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 28 11 00 00 00 00 00 00 01 |..,....(........| 2020-04-10T10:06:15.793 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 1b 92 fa 21 10 00 |.............!..| 2020-04-10T10:06:15.793 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T10:06:15.793 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 15 00 00 00 06 07 00 00 00 73 74 6f |.............sto| 2020-04-10T10:06:15.793 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 70 70 65 64 14 1c 90 5e 6d 07 6e 26 00 |pped...^m.n&.| 2020-04-10T10:06:15.793 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000004d 2020-04-10T10:06:15.793 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.794 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:remote_status_global_b192e243-6abc-405a-b798-b60b9d74691e_620ddb00-0f49-4758-a216-8d9cc890d0fa 2020-04-10T10:06:15.794 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (92 bytes) : 2020-04-10T10:06:15.794 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 2a 11 00 00 00 00 00 00 01 |..,....*........| 2020-04-10T10:06:15.794 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 ed ce d6 f8 10 00 |................| 2020-04-10T10:06:15.794 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T10:06:15.795 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 24 00 00 00 06 16 00 00 00 6c 6f 63 |....$........loc| 2020-04-10T10:06:15.795 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 61 6c 20 69 6d 61 67 65 20 69 73 20 70 72 69 6d |al image is prim| 2020-04-10T10:06:15.795 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 61 72 79 14 1c 90 5e 69 1c de 21 00 |ary...^i..!.| 2020-04-10T10:06:15.795 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000005c 2020-04-10T10:06:15.795 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.795 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:remote_status_global_b583e294-8b69-4e57-9966-783595c454d6_620ddb00-0f49-4758-a216-8d9cc890d0fa 2020-04-10T10:06:15.795 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (77 bytes) : 2020-04-10T10:06:15.796 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 26 11 00 00 00 00 00 00 01 |..,....&........| 2020-04-10T10:06:15.796 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 d5 a9 fe 0b 10 00 |................| 2020-04-10T10:06:15.796 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T10:06:15.796 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 15 00 00 00 06 07 00 00 00 73 74 6f |.............sto| 2020-04-10T10:06:15.796 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 70 70 65 64 13 1c 90 5e 61 9b 79 1f 00 |pped...^a.y..| 2020-04-10T10:06:15.796 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000004d 2020-04-10T10:06:15.796 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.797 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:status_global_003078e2-ed6c-4f36-8bc5-ce0c44ef25d5 2020-04-10T10:06:15.797 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (92 bytes) : 2020-04-10T10:06:15.797 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 9e 11 00 00 00 00 00 00 01 |..,.............| 2020-04-10T10:06:15.797 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 74 d8 db 3f 10 00 |..........t..?..| 2020-04-10T10:06:15.797 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T10:06:15.797 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 24 00 00 00 06 16 00 00 00 6c 6f 63 |....$........loc| 2020-04-10T10:06:15.797 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 61 6c 20 69 6d 61 67 65 20 69 73 20 70 72 69 6d |al image is prim| 2020-04-10T10:06:15.797 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 61 72 79 13 45 90 5e 2c 7a 70 1c 00 |ary.E.^,zp..| 2020-04-10T10:06:15.798 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000005c 2020-04-10T10:06:15.798 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.798 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:status_global_3876de2e-b17d-4c86-94c4-90a11916f44e 2020-04-10T10:06:15.798 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (96 bytes) : 2020-04-10T10:06:15.798 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 9e 11 00 00 00 00 00 00 01 |..,.............| 2020-04-10T10:06:15.798 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 74 d8 db 3f 10 00 |..........t..?..| 2020-04-10T10:06:15.798 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T10:06:15.799 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 28 00 00 00 01 1a 00 00 00 65 72 72 |....(........err| 2020-04-10T10:06:15.800 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 6f 72 20 62 6f 6f 74 73 74 72 61 70 70 69 6e 67 |or bootstrapping| 2020-04-10T10:06:15.800 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 20 72 65 70 6c 61 79 39 1c 90 5e 66 bb b0 03 00 | replay9..^f....| 2020-04-10T10:06:15.800 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000060 2020-04-10T10:06:15.800 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.801 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:status_global_4334771a-4312-4fab-8e12-36d78acc7af1 2020-04-10T10:06:15.801 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (96 bytes) : 2020-04-10T10:06:15.801 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 96 11 00 00 00 00 00 00 01 |..,.............| 2020-04-10T10:06:15.801 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 3e 69 14 eb 10 00 |..........>i....| 2020-04-10T10:06:15.801 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T10:06:15.801 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 28 00 00 00 01 1a 00 00 00 65 72 72 |....(........err| 2020-04-10T10:06:15.802 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 6f 72 20 62 6f 6f 74 73 74 72 61 70 70 69 6e 67 |or bootstrapping| 2020-04-10T10:06:15.802 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 20 72 65 70 6c 61 79 18 1c 90 5e 46 4c d8 39 00 | replay...^FL.9.| 2020-04-10T10:06:15.802 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000060 2020-04-10T10:06:15.802 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.802 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:status_global_4b1961cb-5406-443f-90f1-3627ac4ffffd 2020-04-10T10:06:15.802 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (186 bytes) : 2020-04-10T10:06:15.803 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 96 11 00 00 00 00 00 00 01 |..,.............| 2020-04-10T10:06:15.803 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 3e 69 14 eb 10 00 |..........>i....| 2020-04-10T10:06:15.803 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T10:06:15.803 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 82 00 00 00 04 74 00 00 00 72 65 70 |.........t...rep| 2020-04-10T10:06:15.803 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 6c 61 79 69 6e 67 2c 20 7b 22 72 65 70 6c 61 79 |laying, {"replay| 2020-04-10T10:06:15.803 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 5f 73 74 61 74 65 22 3a 20 22 69 64 6c 65 22 2c |_state": "idle",| 2020-04-10T10:06:15.803 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000060 20 22 72 65 6d 6f 74 65 5f 73 6e 61 70 73 68 6f | "remote_snapsho| 2020-04-10T10:06:15.804 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000070 74 5f 74 69 6d 65 73 74 61 6d 70 22 3a 20 31 35 |t_timestamp": 15| 2020-04-10T10:06:15.804 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000080 38 36 35 30 32 35 39 38 2c 20 22 6c 6f 63 61 6c |86502598, "local| 2020-04-10T10:06:15.804 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000090 5f 73 6e 61 70 73 68 6f 74 5f 74 69 6d 65 73 74 |_snapshot_timest| 2020-04-10T10:06:15.804 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:000000a0 61 6d 70 22 3a 20 31 35 38 36 35 30 32 35 39 38 |amp": 1586502598| 2020-04-10T10:06:15.804 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:000000b0 7d 64 1c 90 5e 8c a7 32 09 00 |}d..^..2..| 2020-04-10T10:06:15.804 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:000000ba 2020-04-10T10:06:15.804 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.805 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:status_global_56419286-13e9-49f6-a20d-2abe6ede083e 2020-04-10T10:06:15.805 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (92 bytes) : 2020-04-10T10:06:15.805 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 9e 11 00 00 00 00 00 00 01 |..,.............| 2020-04-10T10:06:15.805 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 74 d8 db 3f 10 00 |..........t..?..| 2020-04-10T10:06:15.805 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T10:06:15.805 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 24 00 00 00 06 16 00 00 00 6c 6f 63 |....$........loc| 2020-04-10T10:06:15.805 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 61 6c 20 69 6d 61 67 65 20 69 73 20 70 72 69 6d |al image is prim| 2020-04-10T10:06:15.806 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 61 72 79 13 45 90 5e cd d4 35 1c 00 |ary.E.^..5..| 2020-04-10T10:06:15.806 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000005c 2020-04-10T10:06:15.806 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.806 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:status_global_5d3e94e2-1c6c-4146-b461-67ecadf338c4 2020-04-10T10:06:15.806 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (96 bytes) : 2020-04-10T10:06:15.806 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 96 11 00 00 00 00 00 00 01 |..,.............| 2020-04-10T10:06:15.807 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 3e 69 14 eb 10 00 |..........>i....| 2020-04-10T10:06:15.807 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T10:06:15.807 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 28 00 00 00 01 1a 00 00 00 65 72 72 |....(........err| 2020-04-10T10:06:15.807 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 6f 72 20 62 6f 6f 74 73 74 72 61 70 70 69 6e 67 |or bootstrapping| 2020-04-10T10:06:15.807 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 20 72 65 70 6c 61 79 18 1c 90 5e 2d da d8 39 00 | replay...^-..9.| 2020-04-10T10:06:15.807 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000060 2020-04-10T10:06:15.807 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.807 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:status_global_697ea219-b252-4c33-ba20-9b31ea5be82a 2020-04-10T10:06:15.808 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (96 bytes) : 2020-04-10T10:06:15.808 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 9e 11 00 00 00 00 00 00 01 |..,.............| 2020-04-10T10:06:15.808 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 74 d8 db 3f 10 00 |..........t..?..| 2020-04-10T10:06:15.808 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T10:06:15.808 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 28 00 00 00 01 1a 00 00 00 65 72 72 |....(........err| 2020-04-10T10:06:15.808 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 6f 72 20 62 6f 6f 74 73 74 72 61 70 70 69 6e 67 |or bootstrapping| 2020-04-10T10:06:15.808 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 20 72 65 70 6c 61 79 18 1c 90 5e 4f a5 27 3a 00 | replay...^O.':.| 2020-04-10T10:06:15.809 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000060 2020-04-10T10:06:15.809 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.809 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:status_global_70d86755-f73a-4bbe-9493-1b79a191cc0c 2020-04-10T10:06:15.809 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (92 bytes) : 2020-04-10T10:06:15.809 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 96 11 00 00 00 00 00 00 01 |..,.............| 2020-04-10T10:06:15.809 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 3e 69 14 eb 10 00 |..........>i....| 2020-04-10T10:06:15.809 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T10:06:15.810 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 24 00 00 00 06 16 00 00 00 6c 6f 63 |....$........loc| 2020-04-10T10:06:15.810 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 61 6c 20 69 6d 61 67 65 20 69 73 20 70 72 69 6d |al image is prim| 2020-04-10T10:06:15.810 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 61 72 79 5f 1c 90 5e 3b 72 2a 0b 00 |ary_..^;r*..| 2020-04-10T10:06:15.810 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000005c 2020-04-10T10:06:15.810 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.810 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:status_global_79225ce2-9203-4752-bc9b-35a11186acdb 2020-04-10T10:06:15.810 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (96 bytes) : 2020-04-10T10:06:15.811 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 9e 11 00 00 00 00 00 00 01 |..,.............| 2020-04-10T10:06:15.811 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 74 d8 db 3f 10 00 |..........t..?..| 2020-04-10T10:06:15.811 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T10:06:15.811 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 28 00 00 00 01 1a 00 00 00 65 72 72 |....(........err| 2020-04-10T10:06:15.812 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 6f 72 20 62 6f 6f 74 73 74 72 61 70 70 69 6e 67 |or bootstrapping| 2020-04-10T10:06:15.812 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 20 72 65 70 6c 61 79 13 45 90 5e 54 76 aa 1b 00 | replay.E.^Tv...| 2020-04-10T10:06:15.812 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000060 2020-04-10T10:06:15.812 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.812 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:status_global_925c4d02-9086-4edd-b4f3-79c1b7e3734c 2020-04-10T10:06:15.813 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (96 bytes) : 2020-04-10T10:06:15.813 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 96 11 00 00 00 00 00 00 01 |..,.............| 2020-04-10T10:06:15.813 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 3e 69 14 eb 10 00 |..........>i....| 2020-04-10T10:06:15.813 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T10:06:15.813 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 28 00 00 00 01 1a 00 00 00 65 72 72 |....(........err| 2020-04-10T10:06:15.813 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 6f 72 20 62 6f 6f 74 73 74 72 61 70 70 69 6e 67 |or bootstrapping| 2020-04-10T10:06:15.813 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 20 72 65 70 6c 61 79 39 1c 90 5e 8e 30 f9 03 00 | replay9..^.0...| 2020-04-10T10:06:15.814 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000060 2020-04-10T10:06:15.814 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.814 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:status_global_a8183526-4331-4f0d-af4a-7c93c501e927 2020-04-10T10:06:15.814 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (92 bytes) : 2020-04-10T10:06:15.814 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 9e 11 00 00 00 00 00 00 01 |..,.............| 2020-04-10T10:06:15.814 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 74 d8 db 3f 10 00 |..........t..?..| 2020-04-10T10:06:15.814 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T10:06:15.815 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 24 00 00 00 06 16 00 00 00 6c 6f 63 |....$........loc| 2020-04-10T10:06:15.815 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 61 6c 20 69 6d 61 67 65 20 69 73 20 70 72 69 6d |al image is prim| 2020-04-10T10:06:15.815 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 61 72 79 13 45 90 5e 6c 01 71 1c 00 |ary.E.^l.q..| 2020-04-10T10:06:15.815 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000005c 2020-04-10T10:06:15.815 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.815 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:status_global_a9af7d9e-1404-4730-9e18-c0418a915346 2020-04-10T10:06:15.815 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (92 bytes) : 2020-04-10T10:06:15.816 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 9e 11 00 00 00 00 00 00 01 |..,.............| 2020-04-10T10:06:15.816 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 74 d8 db 3f 10 00 |..........t..?..| 2020-04-10T10:06:15.816 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T10:06:15.816 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 24 00 00 00 06 16 00 00 00 6c 6f 63 |....$........loc| 2020-04-10T10:06:15.816 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 61 6c 20 69 6d 61 67 65 20 69 73 20 70 72 69 6d |al image is prim| 2020-04-10T10:06:15.816 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 61 72 79 13 45 90 5e cc 5f 71 1c 00 |ary.E.^._q..| 2020-04-10T10:06:15.817 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000005c 2020-04-10T10:06:15.817 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.817 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:status_global_b192e243-6abc-405a-b798-b60b9d74691e 2020-04-10T10:06:15.817 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (92 bytes) : 2020-04-10T10:06:15.817 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 96 11 00 00 00 00 00 00 01 |..,.............| 2020-04-10T10:06:15.817 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 3e 69 14 eb 10 00 |..........>i....| 2020-04-10T10:06:15.818 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T10:06:15.818 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 24 00 00 00 06 16 00 00 00 6c 6f 63 |....$........loc| 2020-04-10T10:06:15.818 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 61 6c 20 69 6d 61 67 65 20 69 73 20 70 72 69 6d |al image is prim| 2020-04-10T10:06:15.818 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 61 72 79 5f 1c 90 5e dd 76 ec 0a 00 |ary_..^.v...| 2020-04-10T10:06:15.818 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000005c 2020-04-10T10:06:15.818 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.819 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:status_global_b583e294-8b69-4e57-9966-783595c454d6 2020-04-10T10:06:15.819 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (92 bytes) : 2020-04-10T10:06:15.819 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 96 11 00 00 00 00 00 00 01 |..,.............| 2020-04-10T10:06:15.819 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 3e 69 14 eb 10 00 |..........>i....| 2020-04-10T10:06:15.819 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T10:06:15.820 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 24 00 00 00 06 16 00 00 00 6c 6f 63 |....$........loc| 2020-04-10T10:06:15.820 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 61 6c 20 69 6d 61 67 65 20 69 73 20 70 72 69 6d |al image is prim| 2020-04-10T10:06:15.820 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 61 72 79 5f 1c 90 5e 7b f1 2a 0b 00 |ary_..^{.*..| 2020-04-10T10:06:15.820 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000005c 2020-04-10T10:06:15.820 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.820 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.820 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster2 mirror ns1 images 2020-04-10T10:06:15.821 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:15.821 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2 mirror ns1 images 2020-04-10T10:06:15.821 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror --namespace ns1 ls -l 2020-04-10T10:06:15.822 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:15.822 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2 mirrorns1 mirror pool info 2020-04-10T10:06:15.822 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror --namespace ns1 mirror pool info 2020-04-10T10:06:15.823 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.823 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster2 mirrorns1 mirror pool info 2020-04-10T10:06:15.829 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Mode: image 2020-04-10T10:06:15.833 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:15.833 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2 mirrorns1 mirror pool status 2020-04-10T10:06:15.833 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster2 -p mirror --namespace ns1 mirror pool status --verbose 2020-04-10T10:06:15.834 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.834 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster2 mirrorns1 mirror pool status 2020-04-10T10:06:15.878 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:health: OK 2020-04-10T10:06:15.878 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:daemon health: OK 2020-04-10T10:06:15.878 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image health: OK 2020-04-10T10:06:15.878 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:images: 0 total 2020-04-10T10:06:15.879 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.879 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:DAEMONS 2020-04-10T10:06:15.879 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:service 4491: 2020-04-10T10:06:15.879 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: instance_id: 4502 2020-04-10T10:06:15.879 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: client_id: mirror.2 2020-04-10T10:06:15.879 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: hostname: smithi204 2020-04-10T10:06:15.879 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: version: 15.2.0-148-g39357d2 2020-04-10T10:06:15.880 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: leader: false 2020-04-10T10:06:15.880 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: health: OK 2020-04-10T10:06:15.880 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.880 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:service 4499: 2020-04-10T10:06:15.881 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: instance_id: 4510 2020-04-10T10:06:15.881 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: client_id: mirror.3 2020-04-10T10:06:15.881 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: hostname: smithi204 2020-04-10T10:06:15.881 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: version: 15.2.0-148-g39357d2 2020-04-10T10:06:15.881 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: leader: true 2020-04-10T10:06:15.881 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: health: OK 2020-04-10T10:06:15.882 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.882 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.888 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:IMAGES 2020-04-10T10:06:15.888 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:15.888 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror --namespace ns1 ls 2020-04-10T10:06:15.931 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster2 mirror ns1 rbd_mirroring omap vals 2020-04-10T10:06:15.931 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2 mirror ns1 rbd_mirroring omap vals 2020-04-10T10:06:15.931 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rados --cluster cluster2 -p mirror --namespace ns1 listomapvals rbd_mirroring 2020-04-10T10:06:15.967 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:mirror_mode 2020-04-10T10:06:15.968 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (4 bytes) : 2020-04-10T10:06:15.968 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 00 00 00 |....| 2020-04-10T10:06:15.968 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000004 2020-04-10T10:06:15.968 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.968 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:mirror_uuid 2020-04-10T10:06:15.968 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (36 bytes) : 2020-04-10T10:06:15.969 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 36 64 34 64 30 63 65 38 2d 36 39 36 35 2d 34 35 |6d4d0ce8-6965-45| 2020-04-10T10:06:15.969 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 63 32 2d 38 62 64 66 2d 38 38 38 64 36 39 63 39 |c2-8bdf-888d69c9| 2020-04-10T10:06:15.969 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 30 61 35 34 |0a54| 2020-04-10T10:06:15.969 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000024 2020-04-10T10:06:15.969 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.971 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:15.971 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster2 mirror ns2 images 2020-04-10T10:06:15.972 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:15.972 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2 mirror ns2 images 2020-04-10T10:06:15.972 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror --namespace ns2 ls -l 2020-04-10T10:06:16.012 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:16.012 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster2 mirrorns2 mirror pool info 2020-04-10T10:06:16.012 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:16.012 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2 mirrorns2 mirror pool info 2020-04-10T10:06:16.012 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror --namespace ns2 mirror pool info 2020-04-10T10:06:16.054 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Mode: image 2020-04-10T10:06:16.058 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:16.058 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster2 mirrorns2 mirror pool status 2020-04-10T10:06:16.058 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:16.058 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2 mirrorns2 mirror pool status 2020-04-10T10:06:16.058 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster2 -p mirror --namespace ns2 mirror pool status --verbose 2020-04-10T10:06:16.103 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:health: OK 2020-04-10T10:06:16.104 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:daemon health: OK 2020-04-10T10:06:16.104 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image health: OK 2020-04-10T10:06:16.105 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:images: 0 total 2020-04-10T10:06:16.105 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:16.105 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:DAEMONS 2020-04-10T10:06:16.105 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:service 4491: 2020-04-10T10:06:16.105 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: instance_id: 4502 2020-04-10T10:06:16.105 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: client_id: mirror.2 2020-04-10T10:06:16.106 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: hostname: smithi204 2020-04-10T10:06:16.106 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: version: 15.2.0-148-g39357d2 2020-04-10T10:06:16.106 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: leader: false 2020-04-10T10:06:16.106 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: health: OK 2020-04-10T10:06:16.106 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:16.106 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:service 4499: 2020-04-10T10:06:16.106 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: instance_id: 4510 2020-04-10T10:06:16.107 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: client_id: mirror.3 2020-04-10T10:06:16.107 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: hostname: smithi204 2020-04-10T10:06:16.107 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: version: 15.2.0-148-g39357d2 2020-04-10T10:06:16.107 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: leader: true 2020-04-10T10:06:16.107 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: health: OK 2020-04-10T10:06:16.107 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:16.107 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:16.111 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:16.111 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror --namespace ns2 ls 2020-04-10T10:06:16.111 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:IMAGES 2020-04-10T10:06:16.149 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster2 mirror ns2 rbd_mirroring omap vals 2020-04-10T10:06:16.149 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2 mirror ns2 rbd_mirroring omap vals 2020-04-10T10:06:16.150 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rados --cluster cluster2 -p mirror --namespace ns2 listomapvals rbd_mirroring 2020-04-10T10:06:16.183 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:mirror_mode 2020-04-10T10:06:16.183 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (4 bytes) : 2020-04-10T10:06:16.183 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 00 00 00 |....| 2020-04-10T10:06:16.183 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000004 2020-04-10T10:06:16.183 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:16.184 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:mirror_uuid 2020-04-10T10:06:16.184 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (36 bytes) : 2020-04-10T10:06:16.184 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 65 64 32 35 38 30 38 39 2d 34 36 39 34 2d 34 61 |ed258089-4694-4a| 2020-04-10T10:06:16.184 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 62 63 2d 39 65 32 35 2d 63 34 31 62 37 62 62 34 |bc-9e25-c41b7bb4| 2020-04-10T10:06:16.184 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 64 32 39 65 |d29e| 2020-04-10T10:06:16.184 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000024 2020-04-10T10:06:16.185 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:16.185 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:16.186 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster2 mirror_parent images 2020-04-10T10:06:16.186 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:16.186 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2 mirror_parent images 2020-04-10T10:06:16.186 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror_parent --namespace ls -l 2020-04-10T10:06:16.243 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:NAME SIZE PARENT FMT PROT LOCK 2020-04-10T10:06:16.243 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test_parent 128 MiB 2 2020-04-10T10:06:16.243 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test_parent@snap 128 MiB 2 yes 2020-04-10T10:06:16.246 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:16.246 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster2 mirror_parent mirror pool info 2020-04-10T10:06:16.247 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:16.247 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2 mirror_parent mirror pool info 2020-04-10T10:06:16.247 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror_parent --namespace mirror pool info 2020-04-10T10:06:16.286 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Mode: image 2020-04-10T10:06:16.286 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Site Name: cluster2-DNE 2020-04-10T10:06:16.286 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:16.287 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Peer Sites: 2020-04-10T10:06:16.287 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:16.287 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:UUID: 39a2eac0-9f81-4c3b-a329-98e40ae52daf 2020-04-10T10:06:16.287 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Name: cluster1-DNE 2020-04-10T10:06:16.287 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Mirror UUID: fa567d93-f465-4764-93c7-129d993cf8a1 2020-04-10T10:06:16.287 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Direction: rx-tx 2020-04-10T10:06:16.287 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:Client: client.mirror 2020-04-10T10:06:16.290 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:16.290 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster2 mirror_parent mirror pool status 2020-04-10T10:06:16.291 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:16.291 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2 mirror_parent mirror pool status 2020-04-10T10:06:16.291 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster2 -p mirror_parent --namespace mirror pool status --verbose 2020-04-10T10:06:16.333 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:health: WARNING 2020-04-10T10:06:16.333 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:daemon health: OK 2020-04-10T10:06:16.333 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image health: WARNING 2020-04-10T10:06:16.333 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:images: 1 total 2020-04-10T10:06:16.333 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 1 unknown 2020-04-10T10:06:16.334 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:16.334 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:DAEMONS 2020-04-10T10:06:16.334 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:service 4491: 2020-04-10T10:06:16.334 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: instance_id: 2020-04-10T10:06:16.334 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: client_id: mirror.2 2020-04-10T10:06:16.334 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: hostname: smithi204 2020-04-10T10:06:16.334 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: version: 15.2.0-148-g39357d2 2020-04-10T10:06:16.335 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: leader: false 2020-04-10T10:06:16.335 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: health: OK 2020-04-10T10:06:16.335 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:16.335 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:service 4499: 2020-04-10T10:06:16.335 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: instance_id: 4522 2020-04-10T10:06:16.335 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: client_id: mirror.3 2020-04-10T10:06:16.335 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: hostname: smithi204 2020-04-10T10:06:16.335 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: version: 15.2.0-148-g39357d2 2020-04-10T10:06:16.336 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: leader: true 2020-04-10T10:06:16.336 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: health: OK 2020-04-10T10:06:16.336 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:16.336 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:16.366 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:IMAGES 2020-04-10T10:06:16.366 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:test_parent: 2020-04-10T10:06:16.366 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: global_id: dd523e97-79dd-49a7-b2c9-df255e9660d4 2020-04-10T10:06:16.366 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: up+stopped 2020-04-10T10:06:16.367 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: local image is primary 2020-04-10T10:06:16.367 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: service: mirror.3 on smithi204 2020-04-10T10:06:16.367 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10 10:06:11 2020-04-10T10:06:16.367 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: peer_sites: 2020-04-10T10:06:16.367 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: name: cluster1-DNE 2020-04-10T10:06:16.367 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: state: down+unknown 2020-04-10T10:06:16.367 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: description: status not found 2020-04-10T10:06:16.367 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: last_update: 2020-04-10T10:06:16.370 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:16.370 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:16.371 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror_parent --namespace ls 2020-04-10T10:06:16.405 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test_parent info 2020-04-10T10:06:16.405 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test_parent info 2020-04-10T10:06:16.405 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror_parent --namespace info test_parent 2020-04-10T10:06:16.463 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:rbd image 'test_parent': 2020-04-10T10:06:16.463 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: size 128 MiB in 32 objects 2020-04-10T10:06:16.463 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: order 22 (4 MiB objects) 2020-04-10T10:06:16.464 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: snapshot_count: 2 2020-04-10T10:06:16.464 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: id: 125615c533a 2020-04-10T10:06:16.464 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: block_name_prefix: rbd_data.125615c533a 2020-04-10T10:06:16.464 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: format: 2 2020-04-10T10:06:16.464 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: features: layering, exclusive-lock, journaling 2020-04-10T10:06:16.464 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: op_features: 2020-04-10T10:06:16.464 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: flags: 2020-04-10T10:06:16.465 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: create_timestamp: Fri Apr 10 07:09:42 2020 2020-04-10T10:06:16.465 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: access_timestamp: Fri Apr 10 07:09:42 2020 2020-04-10T10:06:16.465 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: modify_timestamp: Fri Apr 10 07:09:42 2020 2020-04-10T10:06:16.465 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: journal: 125615c533a 2020-04-10T10:06:16.465 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring state: enabled 2020-04-10T10:06:16.465 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring mode: snapshot 2020-04-10T10:06:16.465 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring global id: dd523e97-79dd-49a7-b2c9-df255e9660d4 2020-04-10T10:06:16.466 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: mirroring primary: true 2020-04-10T10:06:16.468 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:16.468 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo image test_parent journal status 2020-04-10T10:06:16.468 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror_parent --namespace journal status --image test_parent 2020-04-10T10:06:16.469 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:16.469 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image test_parent journal status 2020-04-10T10:06:16.529 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:minimum_set: 0 2020-04-10T10:06:16.529 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:active_set: 0 2020-04-10T10:06:16.529 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:registered clients: 2020-04-10T10:06:16.529 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: [id=, commit_position=[positions=[[object_number=1, tag_tid=6, entry_tid=29], [object_number=0, tag_tid=6, entry_tid=28], [object_number=3, tag_tid=6, entry_tid=27], [object_number=2, tag_tid=6, entry_tid=26]]], state=connected] 2020-04-10T10:06:16.530 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:16.530 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster2 mirror_parent rbd_mirroring omap vals 2020-04-10T10:06:16.530 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:16.531 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2 mirror_parent rbd_mirroring omap vals 2020-04-10T10:06:16.531 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rados --cluster cluster2 -p mirror_parent --namespace listomapvals rbd_mirroring 2020-04-10T10:06:16.570 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:global_dd523e97-79dd-49a7-b2c9-df255e9660d4 2020-04-10T10:06:16.571 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (15 bytes) : 2020-04-10T10:06:16.571 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 0b 00 00 00 31 32 35 36 31 35 63 35 33 33 61 |....125615c533a| 2020-04-10T10:06:16.571 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000000f 2020-04-10T10:06:16.571 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:16.571 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:image_125615c533a 2020-04-10T10:06:16.571 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (48 bytes) : 2020-04-10T10:06:16.571 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 02 01 2a 00 00 00 24 00 00 00 64 64 35 32 33 65 |..*...$...dd523e| 2020-04-10T10:06:16.572 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 39 37 2d 37 39 64 64 2d 34 39 61 37 2d 62 32 63 |97-79dd-49a7-b2c| 2020-04-10T10:06:16.572 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 39 2d 64 66 32 35 35 65 39 36 36 30 64 34 01 01 |9-df255e9660d4..| 2020-04-10T10:06:16.572 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 2020-04-10T10:06:16.572 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:16.572 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:mirror_mode 2020-04-10T10:06:16.572 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (4 bytes) : 2020-04-10T10:06:16.572 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 00 00 00 |....| 2020-04-10T10:06:16.572 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000004 2020-04-10T10:06:16.573 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:16.573 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:mirror_peer_39a2eac0-9f81-4c3b-a329-98e40ae52daf 2020-04-10T10:06:16.573 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (136 bytes) : 2020-04-10T10:06:16.573 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 02 01 82 00 00 00 24 00 00 00 33 39 61 32 65 61 |......$...39a2ea| 2020-04-10T10:06:16.573 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 63 30 2d 39 66 38 31 2d 34 63 33 62 2d 61 33 32 |c0-9f81-4c3b-a32| 2020-04-10T10:06:16.573 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 39 2d 39 38 65 34 30 61 65 35 32 64 61 66 0c 00 |9-98e40ae52daf..| 2020-04-10T10:06:16.573 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 63 6c 75 73 74 65 72 31 2d 44 4e 45 0d 00 |..cluster1-DNE..| 2020-04-10T10:06:16.574 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 00 00 63 6c 69 65 6e 74 2e 6d 69 72 72 6f 72 ff |..client.mirror.| 2020-04-10T10:06:16.574 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 ff ff ff ff ff ff ff 02 24 00 00 00 66 61 35 36 |........$...fa56| 2020-04-10T10:06:16.574 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000060 37 64 39 33 2d 66 34 36 35 2d 34 37 36 34 2d 39 |7d93-f465-4764-9| 2020-04-10T10:06:16.574 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000070 33 63 37 2d 31 32 39 64 39 39 33 63 66 38 61 31 |3c7-129d993cf8a1| 2020-04-10T10:06:16.574 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000080 0d 1c 90 5e 91 a2 26 30 |...^..&0| 2020-04-10T10:06:16.574 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000088 2020-04-10T10:06:16.574 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:16.574 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:mirror_uuid 2020-04-10T10:06:16.575 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (36 bytes) : 2020-04-10T10:06:16.575 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 65 31 35 31 36 36 31 64 2d 39 61 64 63 2d 34 34 |e151661d-9adc-44| 2020-04-10T10:06:16.575 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 34 38 2d 38 35 37 34 2d 38 39 30 64 32 38 34 37 |48-8574-890d2847| 2020-04-10T10:06:16.575 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 63 37 39 30 |c790| 2020-04-10T10:06:16.575 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000024 2020-04-10T10:06:16.575 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:16.575 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:status_global_dd523e97-79dd-49a7-b2c9-df255e9660d4 2020-04-10T10:06:16.576 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:value (92 bytes) : 2020-04-10T10:06:16.576 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000000 01 01 2c 00 00 00 08 aa 11 00 00 00 00 00 00 01 |..,.............| 2020-04-10T10:06:16.576 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000010 01 01 1c 00 00 00 03 00 00 00 dd 09 3b 03 10 00 |............;...| 2020-04-10T10:06:16.576 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000020 00 00 02 00 00 00 ac 15 0f cc 00 00 00 00 00 00 |................| 2020-04-10T10:06:16.576 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000030 00 00 01 01 24 00 00 00 06 16 00 00 00 6c 6f 63 |....$........loc| 2020-04-10T10:06:16.576 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000040 61 6c 20 69 6d 61 67 65 20 69 73 20 70 72 69 6d |al image is prim| 2020-04-10T10:06:16.576 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:00000050 61 72 79 13 45 90 5e e3 a8 46 23 00 |ary.E.^..F#.| 2020-04-10T10:06:16.577 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:0000005c 2020-04-10T10:06:16.577 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:16.578 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:16.578 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster2 mirror_parent ns1 images 2020-04-10T10:06:16.578 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:16.578 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2 mirror_parent ns1 images 2020-04-10T10:06:16.578 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror_parent --namespace ns1 ls -l 2020-04-10T10:06:16.610 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: namespace 'ns1' does not exist. 2020-04-10T10:06:16.610 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: listing images failed: (2) No such file or directory 2020-04-10T10:06:16.611 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:16.611 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2 mirror_parentns1 mirror pool info 2020-04-10T10:06:16.611 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror_parent --namespace ns1 mirror pool info 2020-04-10T10:06:16.611 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:16.611 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster2 mirror_parentns1 mirror pool info 2020-04-10T10:06:16.640 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: namespace 'ns1' does not exist. 2020-04-10T10:06:16.643 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:16.643 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2 mirror_parentns1 mirror pool status 2020-04-10T10:06:16.643 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster2 -p mirror_parent --namespace ns1 mirror pool status --verbose 2020-04-10T10:06:16.643 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:16.644 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster2 mirror_parentns1 mirror pool status 2020-04-10T10:06:16.676 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: namespace 'ns1' does not exist. 2020-04-10T10:06:16.681 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:16.681 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror_parent --namespace ns1 ls 2020-04-10T10:06:16.681 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:16.719 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster2 mirror_parent ns1 rbd_mirroring omap vals 2020-04-10T10:06:16.719 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2 mirror_parent ns1 rbd_mirroring omap vals 2020-04-10T10:06:16.719 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rados --cluster cluster2 -p mirror_parent --namespace ns1 listomapvals rbd_mirroring 2020-04-10T10:06:16.753 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:error getting omap keys mirror_parent/rbd_mirroring: (2) No such file or directory 2020-04-10T10:06:16.755 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:16.756 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster2 mirror_parent ns2 images 2020-04-10T10:06:16.756 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:16.756 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2 mirror_parent ns2 images 2020-04-10T10:06:16.756 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror_parent --namespace ns2 ls -l 2020-04-10T10:06:16.791 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: namespace 'ns2' does not exist. 2020-04-10T10:06:16.792 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: listing images failed: (2) No such file or directory 2020-04-10T10:06:16.795 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:16.795 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2 mirror_parentns2 mirror pool info 2020-04-10T10:06:16.796 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror_parent --namespace ns2 mirror pool info 2020-04-10T10:06:16.796 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:16.796 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster2 mirror_parentns2 mirror pool info 2020-04-10T10:06:16.832 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: namespace 'ns2' does not exist. 2020-04-10T10:06:16.836 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:16.837 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2 mirror_parentns2 mirror pool status 2020-04-10T10:06:16.837 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ CEPH_ARGS= rbd --cluster cluster2 -p mirror_parent --namespace ns2 mirror pool status --verbose 2020-04-10T10:06:16.837 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:16.837 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster2 mirror_parentns2 mirror pool status 2020-04-10T10:06:16.876 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:rbd: namespace 'ns2' does not exist. 2020-04-10T10:06:16.879 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:16.880 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:16.880 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rbd --cluster cluster2 -p mirror_parent --namespace ns2 ls 2020-04-10T10:06:16.919 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster2 mirror_parent ns2 rbd_mirroring omap vals 2020-04-10T10:06:16.919 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2 mirror_parent ns2 rbd_mirroring omap vals 2020-04-10T10:06:16.919 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ rados --cluster cluster2 -p mirror_parent --namespace ns2 listomapvals rbd_mirroring 2020-04-10T10:06:16.956 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:error getting omap keys mirror_parent/rbd_mirroring: (2) No such file or directory 2020-04-10T10:06:16.959 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:16.959 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local ret 2020-04-10T10:06:16.960 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T10:06:16.960 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:16.961 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_pid_file cluster1:0 2020-04-10T10:06:16.961 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:0 2020-04-10T10:06:16.962 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T10:06:16.962 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:0 cluster instance 2020-04-10T10:06:16.962 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:0 2020-04-10T10:06:16.962 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T10:06:16.962 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T10:06:16.962 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T10:06:16.963 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T10:06:16.963 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster1:0 ] 2020-04-10T10:06:16.963 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T10:06:16.963 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T10:06:16.963 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T10:06:16.963 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T10:06:16.963 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 pid file 2020-04-10T10:06:16.979 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.pid 2020-04-10T10:06:16.980 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pid_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.pid 2020-04-10T10:06:16.980 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ ! -e /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.pid ] 2020-04-10T10:06:16.980 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1 rbd-mirror not running or unknown (/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.pid not exist) 2020-04-10T10:06:16.980 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ continue 2020-04-10T10:06:16.980 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_pid_file cluster1:1 2020-04-10T10:06:16.980 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:1 2020-04-10T10:06:16.980 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T10:06:16.981 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:1 cluster instance 2020-04-10T10:06:16.981 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:1 2020-04-10T10:06:16.981 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T10:06:16.981 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T10:06:16.981 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T10:06:16.981 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T10:06:16.981 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster1:1 ] 2020-04-10T10:06:16.982 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T10:06:16.982 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T10:06:16.982 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T10:06:16.982 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T10:06:16.982 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 pid file 2020-04-10T10:06:16.982 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster1 rbd-mirror not running or unknown (/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.pid not exist) 2020-04-10T10:06:17.002 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.pid 2020-04-10T10:06:17.002 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pid_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.pid 2020-04-10T10:06:17.002 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ ! -e /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.pid ] 2020-04-10T10:06:17.002 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1 rbd-mirror not running or unknown (/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.pid not exist) 2020-04-10T10:06:17.002 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ continue 2020-04-10T10:06:17.003 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_pid_file cluster1:2 2020-04-10T10:06:17.003 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:2 2020-04-10T10:06:17.003 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T10:06:17.003 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:2 cluster instance 2020-04-10T10:06:17.003 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:2 2020-04-10T10:06:17.003 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T10:06:17.003 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T10:06:17.004 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T10:06:17.004 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T10:06:17.004 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster1:2 ] 2020-04-10T10:06:17.004 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T10:06:17.004 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T10:06:17.004 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T10:06:17.004 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T10:06:17.004 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 pid file 2020-04-10T10:06:17.005 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster1 rbd-mirror not running or unknown (/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.pid not exist) 2020-04-10T10:06:17.024 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.pid 2020-04-10T10:06:17.024 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pid_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.pid 2020-04-10T10:06:17.024 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ ! -e /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.pid ] 2020-04-10T10:06:17.025 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1 rbd-mirror not running or unknown (/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.pid not exist) 2020-04-10T10:06:17.025 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ continue 2020-04-10T10:06:17.025 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_pid_file cluster1:3 2020-04-10T10:06:17.025 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster1:3 2020-04-10T10:06:17.025 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T10:06:17.025 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster1:3 cluster instance 2020-04-10T10:06:17.025 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster1:3 2020-04-10T10:06:17.026 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T10:06:17.026 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T10:06:17.026 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T10:06:17.026 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T10:06:17.026 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 3 = cluster1:3 ] 2020-04-10T10:06:17.026 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster1 2020-04-10T10:06:17.026 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster1 2020-04-10T10:06:17.027 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=3 2020-04-10T10:06:17.027 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=3 2020-04-10T10:06:17.027 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.3 pid file 2020-04-10T10:06:17.027 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster1 rbd-mirror not running or unknown (/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.pid not exist) 2020-04-10T10:06:17.037 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.pid 2020-04-10T10:06:17.037 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pid_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.pid 2020-04-10T10:06:17.037 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ ! -e /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.pid ] 2020-04-10T10:06:17.038 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster1 rbd-mirror not running or unknown (/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.pid not exist) 2020-04-10T10:06:17.038 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ continue 2020-04-10T10:06:17.038 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ seq 0 3 2020-04-10T10:06:17.038 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster1 rbd-mirror not running or unknown (/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.pid not exist) 2020-04-10T10:06:17.039 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_pid_file cluster2:0 2020-04-10T10:06:17.039 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2:0 2020-04-10T10:06:17.039 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T10:06:17.039 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:0 cluster instance 2020-04-10T10:06:17.039 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:0 2020-04-10T10:06:17.040 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T10:06:17.040 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T10:06:17.040 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T10:06:17.040 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T10:06:17.040 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 0 = cluster2:0 ] 2020-04-10T10:06:17.040 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster2 2020-04-10T10:06:17.040 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T10:06:17.040 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=0 2020-04-10T10:06:17.041 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=0 2020-04-10T10:06:17.041 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster2 --name client.mirror.0 pid file 2020-04-10T10:06:17.060 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster2-client.mirror.0.pid 2020-04-10T10:06:17.060 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pid_file=/var/run/ceph/rbd-mirror.cluster2-client.mirror.0.pid 2020-04-10T10:06:17.060 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ ! -e /var/run/ceph/rbd-mirror.cluster2-client.mirror.0.pid ] 2020-04-10T10:06:17.061 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2 rbd-mirror not running or unknown (/var/run/ceph/rbd-mirror.cluster2-client.mirror.0.pid not exist) 2020-04-10T10:06:17.061 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ continue 2020-04-10T10:06:17.061 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_pid_file cluster2:1 2020-04-10T10:06:17.061 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2:1 2020-04-10T10:06:17.061 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T10:06:17.061 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:1 cluster instance 2020-04-10T10:06:17.061 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:1 2020-04-10T10:06:17.062 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T10:06:17.062 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T10:06:17.062 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T10:06:17.062 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T10:06:17.062 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 1 = cluster2:1 ] 2020-04-10T10:06:17.063 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster2 2020-04-10T10:06:17.063 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T10:06:17.063 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=1 2020-04-10T10:06:17.063 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=1 2020-04-10T10:06:17.063 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster2 --name client.mirror.1 pid file 2020-04-10T10:06:17.063 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster2 rbd-mirror not running or unknown (/var/run/ceph/rbd-mirror.cluster2-client.mirror.0.pid not exist) 2020-04-10T10:06:17.078 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster2-client.mirror.1.pid 2020-04-10T10:06:17.078 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pid_file=/var/run/ceph/rbd-mirror.cluster2-client.mirror.1.pid 2020-04-10T10:06:17.078 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ ! -e /var/run/ceph/rbd-mirror.cluster2-client.mirror.1.pid ] 2020-04-10T10:06:17.079 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2 rbd-mirror not running or unknown (/var/run/ceph/rbd-mirror.cluster2-client.mirror.1.pid not exist) 2020-04-10T10:06:17.079 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ continue 2020-04-10T10:06:17.079 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_pid_file cluster2:2 2020-04-10T10:06:17.079 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2:2 2020-04-10T10:06:17.079 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T10:06:17.080 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:2 cluster instance 2020-04-10T10:06:17.080 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:2 2020-04-10T10:06:17.080 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=cluster 2020-04-10T10:06:17.080 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T10:06:17.080 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T10:06:17.080 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T10:06:17.080 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster2:2 ] 2020-04-10T10:06:17.081 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval cluster=cluster2 2020-04-10T10:06:17.081 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T10:06:17.081 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T10:06:17.081 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T10:06:17.081 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster2 --name client.mirror.2 pid file 2020-04-10T10:06:17.081 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster2 rbd-mirror not running or unknown (/var/run/ceph/rbd-mirror.cluster2-client.mirror.1.pid not exist) 2020-04-10T10:06:17.096 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster2-client.mirror.2.pid 2020-04-10T10:06:17.097 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pid_file=/var/run/ceph/rbd-mirror.cluster2-client.mirror.2.pid 2020-04-10T10:06:17.097 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ ! -e /var/run/ceph/rbd-mirror.cluster2-client.mirror.2.pid ] 2020-04-10T10:06:17.097 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local pid 2020-04-10T10:06:17.097 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cat /var/run/ceph/rbd-mirror.cluster2-client.mirror.2.pid 2020-04-10T10:06:17.097 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ pid=19197 2020-04-10T10:06:17.097 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ -z 19197 ] 2020-04-10T10:06:17.098 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo rbd-mirror process in ps output: 2020-04-10T10:06:17.098 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ awk -v pid=19197 NR == 1 {print} $2 == pid {print; exit 1} 2020-04-10T10:06:17.098 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ps auxww 2020-04-10T10:06:17.098 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: rbd-mirror process in ps output: 2020-04-10T10:06:17.107 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND 2020-04-10T10:06:17.107 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:ubuntu 19197 0.1 0.2 992832 88788 ? Ssl 07:08 0:16 rbd-mirror --cluster cluster2 --id mirror.2 --rbd-mirror-delete-retry-interval=5 --rbd-mirror-image-state-check-interval=5 --rbd-mirror-journal-poll-age=1 --rbd-mirror-pool-replayers-refresh-interval=5 --debug-rbd=30 --debug-journaler=30 --debug-rbd_mirror=30 --daemonize=true 2020-04-10T10:06:17.109 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout: 2020-04-10T10:06:17.109 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo 2020-04-10T10:06:17.110 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ daemon_asok_file cluster2:2 cluster2 2020-04-10T10:06:17.110 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local local_cluster=cluster2:2 2020-04-10T10:06:17.110 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster=cluster2 2020-04-10T10:06:17.110 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance 2020-04-10T10:06:17.110 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ set_cluster_instance cluster2:2 local_cluster instance 2020-04-10T10:06:17.110 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local val=cluster2:2 2020-04-10T10:06:17.110 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local cluster_var_name=local_cluster 2020-04-10T10:06:17.111 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local instance_var_name=instance 2020-04-10T10:06:17.111 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ cluster=cluster2 2020-04-10T10:06:17.111 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T10:06:17.111 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ 2 = cluster2:2 ] 2020-04-10T10:06:17.111 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval local_cluster=cluster2 2020-04-10T10:06:17.111 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local_cluster=cluster2 2020-04-10T10:06:17.111 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ eval instance=2 2020-04-10T10:06:17.111 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ instance=2 2020-04-10T10:06:17.112 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph-conf --cluster cluster2 --name client.mirror.2 admin socket 2020-04-10T10:06:17.128 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo /var/run/ceph/rbd-mirror.cluster2-client.mirror.2.asok 2020-04-10T10:06:17.128 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster2-client.mirror.2.asok 2020-04-10T10:06:17.129 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ [ ! -S /var/run/ceph/rbd-mirror.cluster2-client.mirror.2.asok ] 2020-04-10T10:06:17.129 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ echo cluster2 rbd-mirror status 2020-04-10T10:06:17.129 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster2-client.mirror.2.asok rbd mirror status 2020-04-10T10:06:17.129 INFO:tasks.workunit.cluster1.client.mirror.smithi204.stdout:cluster2 rbd-mirror status 2020-04-10T10:12:06.481 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T10:12:06.474+0000 7fefd66bd700 -1 reset not still connected to 0x556b42443860 2020-04-10T10:12:06.482 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T10:12:06.478+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283aa90 2020-04-10T10:12:06.483 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T10:12:06.478+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283b930 2020-04-10T10:12:06.483 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T10:12:06.478+0000 7fefd66bd700 -1 reset not still connected to 0x556b4313a820 2020-04-10T10:12:21.493 INFO:tasks.ceph.cluster1.osd.2.smithi101.stderr:2020-04-10T10:12:21.488+0000 7f8105502700 -1 reset not still connected to 0x55d7e67c2000 2020-04-10T10:13:11.493 INFO:tasks.ceph.cluster1.osd.2.smithi101.stderr:2020-04-10T10:13:11.479+0000 7f8105502700 -1 reset not still connected to 0x55d7e67c2000 2020-04-10T10:26:51.514 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T10:26:51.509+0000 7fefd66bd700 -1 reset not still connected to 0x556b42443860 2020-04-10T10:26:51.514 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T10:26:51.509+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283aa90 2020-04-10T10:26:51.515 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T10:26:51.509+0000 7fefd66bd700 -1 reset not still connected to 0x556b4313a820 2020-04-10T10:29:26.511 INFO:tasks.ceph.cluster1.osd.2.smithi101.stderr:2020-04-10T10:29:26.504+0000 7f8105502700 -1 reset not still connected to 0x55d7e67c2000 2020-04-10T10:50:11.561 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T10:50:11.552+0000 7fefd66bd700 -1 reset not still connected to 0x556b42443860 2020-04-10T10:50:11.569 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T10:50:11.552+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283aa90 2020-04-10T10:50:11.570 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T10:50:11.552+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283b930 2020-04-10T10:50:11.570 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T10:50:11.552+0000 7fefd66bd700 -1 reset not still connected to 0x556b4313a820 2020-04-10T11:05:56.573 INFO:tasks.ceph.cluster1.osd.2.smithi101.stderr:2020-04-10T11:05:56.565+0000 7f8105502700 -1 reset not still connected to 0x55d7e67c2000 2020-04-10T11:15:31.601 INFO:tasks.ceph.cluster1.osd.2.smithi101.stderr:2020-04-10T11:15:31.592+0000 7f8105502700 -1 reset not still connected to 0x55d7e67c2000 2020-04-10T11:19:06.605 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T11:19:06.598+0000 7fefd66bd700 -1 reset not still connected to 0x556b42443860 2020-04-10T11:19:06.606 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T11:19:06.602+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283aa90 2020-04-10T11:19:06.608 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T11:19:06.602+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283b930 2020-04-10T11:19:06.608 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T11:19:06.602+0000 7fefd66bd700 -1 reset not still connected to 0x556b4313a820 2020-04-10T11:20:36.718 INFO:tasks.ceph.cluster2.osd.1.smithi204.stderr:2020-04-10T11:20:36.712+0000 7f5dcede3700 -1 reset not still connected to 0x559a9f61e080 2020-04-10T11:21:36.599 INFO:tasks.ceph.cluster1.osd.2.smithi101.stderr:2020-04-10T11:21:36.595+0000 7f8105502700 -1 reset not still connected to 0x55d7e67c2000 2020-04-10T11:29:06.616 INFO:tasks.ceph.cluster1.osd.2.smithi101.stderr:2020-04-10T11:29:06.608+0000 7f8105502700 -1 reset not still connected to 0x55d7e67c2000 2020-04-10T11:29:56.736 INFO:tasks.ceph.cluster2.osd.1.smithi204.stderr:2020-04-10T11:29:56.728+0000 7f5dcede3700 -1 reset not still connected to 0x559a9f61e080 2020-04-10T11:32:46.620 INFO:tasks.ceph.cluster1.osd.2.smithi101.stderr:2020-04-10T11:32:46.611+0000 7f8105502700 -1 reset not still connected to 0x55d7e67c2000 2020-04-10T11:39:06.749 INFO:tasks.ceph.cluster2.osd.1.smithi204.stderr:2020-04-10T11:39:06.740+0000 7f5dcede3700 -1 reset not still connected to 0x559a9f61e080 2020-04-10T11:43:31.640 INFO:tasks.ceph.cluster1.osd.2.smithi101.stderr:2020-04-10T11:43:31.632+0000 7f8105502700 -1 reset not still connected to 0x55d7e67c2000 2020-04-10T11:47:06.645 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T11:47:06.636+0000 7fefd66bd700 -1 reset not still connected to 0x556b42443860 2020-04-10T11:47:06.663 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T11:47:06.636+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283aa90 2020-04-10T11:47:06.663 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T11:47:06.636+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283b930 2020-04-10T11:47:06.663 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T11:47:06.636+0000 7fefd66bd700 -1 reset not still connected to 0x556b4313a820 2020-04-10T11:47:11.645 INFO:tasks.ceph.cluster1.osd.2.smithi101.stderr:2020-04-10T11:47:11.639+0000 7f8105502700 -1 reset not still connected to 0x55d7e67c2000 2020-04-10T11:49:01.653 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T11:49:01.647+0000 7fefd66bd700 -1 reset not still connected to 0x556b42443860 2020-04-10T11:49:01.655 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T11:49:01.651+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283aa90 2020-04-10T11:49:01.655 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T11:49:01.651+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283b930 2020-04-10T11:49:01.655 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T11:49:01.651+0000 7fefd66bd700 -1 reset not still connected to 0x556b4313a820 2020-04-10T11:49:56.650 INFO:tasks.ceph.cluster1.osd.2.smithi101.stderr:2020-04-10T11:49:56.647+0000 7f8105502700 -1 reset not still connected to 0x55d7e67c2000 2020-04-10T12:00:06.785 INFO:tasks.ceph.cluster2.osd.1.smithi204.stderr:2020-04-10T12:00:06.777+0000 7f5dcede3700 -1 reset not still connected to 0x559a9f61e080 2020-04-10T12:02:51.674 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T12:02:51.670+0000 7fefd66bd700 -1 reset not still connected to 0x556b42443860 2020-04-10T12:02:51.674 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T12:02:51.670+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283aa90 2020-04-10T12:02:51.677 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T12:02:51.670+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283b930 2020-04-10T12:02:51.677 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T12:02:51.670+0000 7fefd66bd700 -1 reset not still connected to 0x556b4313a820 2020-04-10T12:12:56.682 INFO:tasks.ceph.cluster1.osd.2.smithi101.stderr:2020-04-10T12:12:56.679+0000 7f8105502700 -1 reset not still connected to 0x55d7e67c2000 2020-04-10T12:16:06.711 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T12:16:06.706+0000 7fefd66bd700 -1 reset not still connected to 0x556b42443860 2020-04-10T12:16:06.712 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T12:16:06.706+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283aa90 2020-04-10T12:16:06.713 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T12:16:06.706+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283b930 2020-04-10T12:16:06.713 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T12:16:06.706+0000 7fefd66bd700 -1 reset not still connected to 0x556b4313a820 2020-04-10T12:19:11.707 INFO:tasks.ceph.cluster1.osd.2.smithi101.stderr:2020-04-10T12:19:11.702+0000 7f8105502700 -1 reset not still connected to 0x55d7e67c2000 2020-04-10T12:20:56.706 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T12:20:56.698+0000 7fefd66bd700 -1 reset not still connected to 0x556b42443860 2020-04-10T12:20:56.706 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T12:20:56.702+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283aa90 2020-04-10T12:20:56.709 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T12:20:56.702+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283b930 2020-04-10T12:20:56.709 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T12:20:56.702+0000 7fefd66bd700 -1 reset not still connected to 0x556b4313a820 2020-04-10T12:24:06.710 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T12:24:06.707+0000 7fefd66bd700 -1 reset not still connected to 0x556b42443860 2020-04-10T12:24:06.736 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T12:24:06.707+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283aa90 2020-04-10T12:24:06.736 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T12:24:06.707+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283b930 2020-04-10T12:24:06.736 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T12:24:06.707+0000 7fefd66bd700 -1 reset not still connected to 0x556b4313a820 2020-04-10T12:27:36.709 INFO:tasks.ceph.cluster1.osd.2.smithi101.stderr:2020-04-10T12:27:36.702+0000 7f8105502700 -1 reset not still connected to 0x55d7e67c2000 2020-04-10T12:29:46.835 INFO:tasks.ceph.cluster2.osd.1.smithi204.stderr:2020-04-10T12:29:46.830+0000 7f5dcede3700 -1 reset not still connected to 0x559a9f61e080 2020-04-10T12:29:56.719 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T12:29:56.714+0000 7fefd66bd700 -1 reset not still connected to 0x556b42443860 2020-04-10T12:29:56.720 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T12:29:56.714+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283aa90 2020-04-10T12:29:56.721 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T12:29:56.714+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283b930 2020-04-10T12:29:56.721 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T12:29:56.714+0000 7fefd66bd700 -1 reset not still connected to 0x556b4313a820 2020-04-10T12:57:47.653 INFO:tasks.ceph.cluster1.osd.2.smithi101.stderr:2020-04-10T12:57:47.643+0000 7f8105502700 -1 reset not still connected to 0x55d7e641d260 2020-04-10T12:59:36.882 INFO:tasks.ceph.cluster2.osd.1.smithi204.stderr:2020-04-10T12:59:36.878+0000 7f5dcede3700 -1 reset not still connected to 0x559a9f61e080 2020-04-10T13:12:21.780 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T13:12:21.775+0000 7fefd66bd700 -1 reset not still connected to 0x556b42443860 2020-04-10T13:12:21.796 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T13:12:21.779+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283aa90 2020-04-10T13:12:21.796 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T13:12:21.779+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283b930 2020-04-10T13:12:21.797 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T13:12:21.779+0000 7fefd66bd700 -1 reset not still connected to 0x556b4313a820 2020-04-10T13:32:26.937 INFO:tasks.ceph.cluster2.osd.1.smithi204.stderr:2020-04-10T13:32:26.928+0000 7f5dcede3700 -1 reset not still connected to 0x559a9f61e080 2020-04-10T13:32:51.810 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T13:32:51.800+0000 7fefd66bd700 -1 reset not still connected to 0x556b42443860 2020-04-10T13:32:51.810 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T13:32:51.804+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283aa90 2020-04-10T13:32:51.840 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T13:32:51.832+0000 7fefd66bd700 -1 reset not still connected to 0x556b42443860 2020-04-10T13:32:51.840 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T13:32:51.832+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283aa90 2020-04-10T13:32:51.851 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T13:32:51.832+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283b930 2020-04-10T13:32:51.851 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T13:32:51.832+0000 7fefd66bd700 -1 reset not still connected to 0x556b4313a820 2020-04-10T13:37:11.832 INFO:tasks.ceph.cluster1.osd.2.smithi101.stderr:2020-04-10T13:37:11.824+0000 7f8105502700 -1 reset not still connected to 0x55d7e67c2000 2020-04-10T13:48:45.098 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T13:48:45.090+0000 7fefd66bd700 -1 reset not still connected to 0x556b3e67fe10 2020-04-10T13:48:45.124 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T13:48:45.090+0000 7fefd66bd700 -1 reset not still connected to 0x556b42443860 2020-04-10T13:48:45.124 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T13:48:45.090+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283aa90 2020-04-10T13:51:31.851 INFO:tasks.ceph.cluster1.osd.2.smithi101.stderr:2020-04-10T13:51:31.843+0000 7f8105502700 -1 reset not still connected to 0x55d7e67c2000 2020-04-10T14:05:35.844 INFO:tasks.ceph.cluster1.osd.2.smithi101.stderr:2020-04-10T14:05:35.835+0000 7f8105502700 -1 reset not still connected to 0x55d7e641d260 2020-04-10T14:06:06.887 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T14:06:06.882+0000 7fefd66bd700 -1 reset not still connected to 0x556b42443860 2020-04-10T14:06:06.889 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T14:06:06.882+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283aa90 2020-04-10T14:06:06.889 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T14:06:06.882+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283b930 2020-04-10T14:06:06.889 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T14:06:06.882+0000 7fefd66bd700 -1 reset not still connected to 0x556b4313a820 2020-04-10T14:06:16.886 INFO:tasks.ceph.cluster1.osd.2.smithi101.stderr:2020-04-10T14:06:16.882+0000 7f8105502700 -1 reset not still connected to 0x55d7e67c2000 2020-04-10T14:36:06.926 INFO:tasks.ceph.cluster1.osd.2.smithi101.stderr:2020-04-10T14:36:06.916+0000 7f8105502700 -1 reset not still connected to 0x55d7e67c2000 2020-04-10T14:36:22.045 INFO:tasks.ceph.cluster2.osd.1.smithi204.stderr:2020-04-10T14:36:22.042+0000 7f5dcede3700 -1 reset not still connected to 0x559a9f61e080 2020-04-10T14:40:16.935 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T14:40:16.931+0000 7fefd66bd700 -1 reset not still connected to 0x556b42443860 2020-04-10T14:40:16.935 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T14:40:16.931+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283aa90 2020-04-10T14:41:56.943 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T14:41:56.937+0000 7fefd66bd700 -1 reset not still connected to 0x556b42443860 2020-04-10T14:41:56.945 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T14:41:56.941+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283aa90 2020-04-10T14:41:56.947 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T14:41:56.941+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283b930 2020-04-10T14:41:56.947 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T14:41:56.941+0000 7fefd66bd700 -1 reset not still connected to 0x556b4313a820 2020-04-10T14:50:26.950 INFO:tasks.ceph.cluster1.osd.2.smithi101.stderr:2020-04-10T14:50:26.944+0000 7f8105502700 -1 reset not still connected to 0x55d7e67c2000 2020-04-10T14:56:01.966 INFO:tasks.ceph.cluster1.osd.2.smithi101.stderr:2020-04-10T14:56:01.960+0000 7f8105502700 -1 reset not still connected to 0x55d7e67c2000 2020-04-10T15:03:41.978 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T15:03:41.973+0000 7fefd66bd700 -1 reset not still connected to 0x556b42443860 2020-04-10T15:03:41.991 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T15:03:41.973+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283aa90 2020-04-10T15:03:41.992 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T15:03:41.973+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283b930 2020-04-10T15:03:41.992 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T15:03:41.973+0000 7fefd66bd700 -1 reset not still connected to 0x556b4313a820 2020-04-10T15:15:28.606 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T15:15:28.603+0000 7fefd66bd700 -1 reset not still connected to 0x556b3e681670 2020-04-10T15:15:28.615 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T15:15:28.603+0000 7fefd66bd700 -1 reset not still connected to 0x556b42444700 2020-04-10T15:15:28.615 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T15:15:28.603+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283bd40 2020-04-10T15:16:52.005 INFO:tasks.ceph.cluster1.osd.2.smithi101.stderr:2020-04-10T15:16:51.999+0000 7f8105502700 -1 reset not still connected to 0x55d7e67c2000 2020-04-10T15:44:35.457 INFO:tasks.ceph.cluster2.osd.0.smithi204.stderr:2020-04-10T15:44:35.432+0000 7f034447a700 -1 reset not still connected to 0x55c293449380 2020-04-10T15:44:35.643 INFO:tasks.ceph.cluster2.osd.0.smithi204.stderr:2020-04-10T15:44:35.436+0000 7f034447a700 -1 reset not still connected to 0x55c29470a5b0 2020-04-10T15:47:22.046 INFO:tasks.ceph.cluster1.osd.2.smithi101.stderr:2020-04-10T15:47:22.040+0000 7f8105502700 -1 reset not still connected to 0x55d7e67c2000 2020-04-10T15:47:57.051 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T15:47:57.045+0000 7fefd66bd700 -1 reset not still connected to 0x556b42443860 2020-04-10T15:47:57.053 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T15:47:57.045+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283aa90 2020-04-10T15:47:57.055 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T15:47:57.049+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283b930 2020-04-10T15:47:57.055 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T15:47:57.049+0000 7fefd66bd700 -1 reset not still connected to 0x556b4313a820 2020-04-10T15:53:37.067 INFO:tasks.ceph.cluster1.osd.2.smithi101.stderr:2020-04-10T15:53:37.059+0000 7f8105502700 -1 reset not still connected to 0x55d7e67c2000 2020-04-10T16:03:52.077 INFO:tasks.ceph.cluster1.osd.2.smithi101.stderr:2020-04-10T16:03:52.069+0000 7f8105502700 -1 reset not still connected to 0x55d7e67c2000 2020-04-10T16:05:12.077 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T16:05:12.070+0000 7fefd66bd700 -1 reset not still connected to 0x556b42443860 2020-04-10T16:05:12.077 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T16:05:12.070+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283aa90 2020-04-10T16:05:12.077 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T16:05:12.070+0000 7fefd66bd700 -1 reset not still connected to 0x556b4313a820 2020-04-10T16:09:02.200 INFO:tasks.ceph.cluster2.osd.1.smithi204.stderr:2020-04-10T16:09:02.194+0000 7f5dcede3700 -1 reset not still connected to 0x559a9f61e080 2020-04-10T16:10:37.096 INFO:tasks.ceph.cluster1.osd.2.smithi101.stderr:2020-04-10T16:10:37.088+0000 7f8105502700 -1 reset not still connected to 0x55d7e67c2000 2020-04-10T16:17:27.115 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T16:17:27.111+0000 7fefd66bd700 -1 reset not still connected to 0x556b42443860 2020-04-10T16:17:27.115 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T16:17:27.111+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283aa90 2020-04-10T16:17:27.118 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T16:17:27.115+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283b930 2020-04-10T16:17:27.118 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T16:17:27.115+0000 7fefd66bd700 -1 reset not still connected to 0x556b4313a820 2020-04-10T16:18:07.112 INFO:tasks.ceph.cluster1.osd.2.smithi101.stderr:2020-04-10T16:18:07.105+0000 7f8105502700 -1 reset not still connected to 0x55d7e67c2000 2020-04-10T16:26:17.111 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T16:26:17.107+0000 7fefd66bd700 -1 reset not still connected to 0x556b42443860 2020-04-10T16:26:17.119 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T16:26:17.107+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283aa90 2020-04-10T16:26:17.119 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T16:26:17.111+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283b930 2020-04-10T16:26:17.119 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T16:26:17.111+0000 7fefd66bd700 -1 reset not still connected to 0x556b4313a820 2020-04-10T16:27:17.127 INFO:tasks.ceph.cluster1.osd.2.smithi101.stderr:2020-04-10T16:27:17.120+0000 7f8105502700 -1 reset not still connected to 0x55d7e67c2000 2020-04-10T16:29:57.131 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T16:29:57.123+0000 7fefd66bd700 -1 reset not still connected to 0x556b42443860 2020-04-10T16:29:57.131 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T16:29:57.123+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283aa90 2020-04-10T16:29:57.133 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T16:29:57.123+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283b930 2020-04-10T16:29:57.133 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T16:29:57.123+0000 7fefd66bd700 -1 reset not still connected to 0x556b4313a820 2020-04-10T16:33:32.137 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T16:33:32.128+0000 7fefd66bd700 -1 reset not still connected to 0x556b42443860 2020-04-10T16:33:32.140 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T16:33:32.132+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283aa90 2020-04-10T16:33:32.140 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T16:33:32.132+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283b930 2020-04-10T16:33:32.141 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T16:33:32.132+0000 7fefd66bd700 -1 reset not still connected to 0x556b4313a820 2020-04-10T16:36:02.128 INFO:tasks.ceph.cluster2.osd.0.smithi204.stderr:2020-04-10T16:36:02.122+0000 7f034447a700 -1 reset not still connected to 0x55c293449e10 2020-04-10T16:36:02.129 INFO:tasks.ceph.cluster2.osd.0.smithi204.stderr:2020-04-10T16:36:02.122+0000 7f034447a700 -1 reset not still connected to 0x55c29470ab60 2020-04-10T16:36:02.129 INFO:tasks.ceph.cluster2.osd.0.smithi204.stderr:2020-04-10T16:36:02.122+0000 7f034447a700 -1 reset not still connected to 0x55c29470b520 2020-04-10T16:36:02.129 INFO:tasks.ceph.cluster2.osd.0.smithi204.stderr:2020-04-10T16:36:02.122+0000 7f034447a700 -1 reset not still connected to 0x55c294d55450 2020-04-10T16:40:12.255 INFO:tasks.ceph.cluster2.osd.1.smithi204.stderr:2020-04-10T16:40:12.250+0000 7f5dcede3700 -1 reset not still connected to 0x559a9f61e080 2020-04-10T16:44:27.146 INFO:tasks.ceph.cluster1.osd.2.smithi101.stderr:2020-04-10T16:44:27.141+0000 7f8105502700 -1 reset not still connected to 0x55d7e67c2000 2020-04-10T16:51:47.164 INFO:tasks.ceph.cluster1.osd.2.smithi101.stderr:2020-04-10T16:51:47.158+0000 7f8105502700 -1 reset not still connected to 0x55d7e67c2000 2020-04-10T16:53:17.150 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T16:53:17.146+0000 7fefd66bd700 -1 reset not still connected to 0x556b42443860 2020-04-10T16:53:17.153 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T16:53:17.150+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283aa90 2020-04-10T16:53:17.153 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T16:53:17.150+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283b930 2020-04-10T16:53:17.153 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T16:53:17.150+0000 7fefd66bd700 -1 reset not still connected to 0x556b4313a820 2020-04-10T16:54:27.169 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T16:54:27.165+0000 7fefd66bd700 -1 reset not still connected to 0x556b42443860 2020-04-10T16:54:27.178 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T16:54:27.165+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283aa90 2020-04-10T16:54:27.178 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T16:54:27.165+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283b930 2020-04-10T16:54:27.178 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T16:54:27.165+0000 7fefd66bd700 -1 reset not still connected to 0x556b4313a820 2020-04-10T16:55:32.171 INFO:tasks.ceph.cluster1.osd.2.smithi101.stderr:2020-04-10T16:55:32.165+0000 7f8105502700 -1 reset not still connected to 0x55d7e67c2000 2020-04-10T17:07:52.190 INFO:tasks.ceph.cluster1.osd.2.smithi101.stderr:2020-04-10T17:07:52.184+0000 7f8105502700 -1 reset not still connected to 0x55d7e67c2000 2020-04-10T17:11:17.194 INFO:tasks.ceph.cluster1.osd.2.smithi101.stderr:2020-04-10T17:11:17.187+0000 7f8105502700 -1 reset not still connected to 0x55d7e67c2000 2020-04-10T17:12:27.189 INFO:tasks.ceph.cluster1.osd.2.smithi101.stderr:2020-04-10T17:12:27.181+0000 7f8105502700 -1 reset not still connected to 0x55d7e67c2000 2020-04-10T17:18:12.319 INFO:tasks.ceph.cluster2.osd.1.smithi204.stderr:2020-04-10T17:18:12.312+0000 7f5dcede3700 -1 reset not still connected to 0x559a9f61e080 2020-04-10T17:50:22.373 INFO:tasks.ceph.cluster2.osd.1.smithi204.stderr:2020-04-10T17:50:22.371+0000 7f5dcede3700 -1 reset not still connected to 0x559a9f61e080 2020-04-10T17:53:21.555 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T17:53:21.548+0000 7fefd66bd700 -1 reset not still connected to 0x556b3e67fe10 2020-04-10T17:53:21.555 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T17:53:21.548+0000 7fefd66bd700 -1 reset not still connected to 0x556b42443860 2020-04-10T17:53:21.555 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T17:53:21.548+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283aa90 2020-04-10T17:57:07.385 INFO:tasks.ceph.cluster2.osd.1.smithi204.stderr:2020-04-10T17:57:07.377+0000 7f5dcede3700 -1 reset not still connected to 0x559a9f61e080 2020-04-10T17:59:57.277 INFO:tasks.ceph.cluster1.osd.2.smithi101.stderr:2020-04-10T17:59:57.271+0000 7f8105502700 -1 reset not still connected to 0x55d7e67c2000 2020-04-10T18:08:17.288 INFO:tasks.ceph.cluster1.osd.2.smithi101.stderr:2020-04-10T18:08:17.280+0000 7f8105502700 -1 reset not still connected to 0x55d7e67c2000 2020-04-10T18:12:37.287 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T18:12:37.283+0000 7fefd66bd700 -1 reset not still connected to 0x556b42443860 2020-04-10T18:12:37.301 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T18:12:37.283+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283aa90 2020-04-10T18:33:37.337 INFO:tasks.ceph.cluster1.osd.2.smithi101.stderr:2020-04-10T18:33:37.332+0000 7f8105502700 -1 reset not still connected to 0x55d7e67c2000 2020-04-10T18:33:47.338 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T18:33:47.332+0000 7fefd66bd700 -1 reset not still connected to 0x556b42443860 2020-04-10T18:33:47.339 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T18:33:47.332+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283aa90 2020-04-10T18:33:47.340 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T18:33:47.332+0000 7fefd66bd700 -1 reset not still connected to 0x556b4283b930 2020-04-10T18:33:47.340 INFO:tasks.ceph.cluster2.osd.2.smithi204.stderr:2020-04-10T18:33:47.332+0000 7fefd66bd700 -1 reset not still connected to 0x556b4313a820 2020-04-10T18:38:12.450 INFO:tasks.ceph.cluster2.osd.1.smithi204.stderr:2020-04-10T18:38:12.444+0000 7f5dcede3700 -1 reset not still connected to 0x559a9f61e080 2020-04-10T18:49:33.983 DEBUG:teuthology.exit:Got signal 15; running 2 handlers... 2020-04-10T18:49:34.007 DEBUG:teuthology.task.console_log:Killing console logger for smithi204 2020-04-10T18:49:34.007 DEBUG:teuthology.task.console_log:Killing console logger for smithi101 2020-04-10T18:49:34.007 DEBUG:teuthology.task.console_log:Killing console logger for smithi204 2020-04-10T18:49:34.008 DEBUG:teuthology.task.console_log:Killing console logger for smithi101 2020-04-10T18:49:34.008 DEBUG:teuthology.exit:Finished running handlers