CernVM Testing

Completed

• Followed Yushu's tutorial at CernVMTutorialHead on VirtualBox up to running the athena hello world example
• Get panda pilot running on CernVM. See PandaPilotCloudRecipe
• LxCloud results: LxCloudHCAtlasTesting
• Helix Nebula Science Cloud:
  • CloudSigmaConfig, CloudSigmaResults

TODO

• Grid of Clouds ideas:
  • cvm-based management tool:
    • cvm will instantiate and contextualize a VM at any EC2 API site. cvm will pass the grid credentials and start a job, e.g. the panda pilot.
    • what's missing inside the VM: watchdog to monitor the VMs to ensure that something is still running -- shutdown otherwise
    • what's missing outside the VM: need a tool to wrap around cvm allowing operator to define a number of instances to run at a number of different sites. E.g. one could have a config file saying: cern_analy:100,vic_analy:50 and this would instantiate 100 nodes at CERN, 50 in Victoria, and it would run periodically to instantiate new VMs in case they were shut down (e.g. credential expired). In future it would dynamically decide how many nodes to run based on the workload, e.g. on the number of jobs in the panda queue.
    • also a status command would list how many instances running at each site.
  • Victoria cloud scheduler
    • with "cloud scheduler" one can submit to a condor queue along with a vm image description and the cloud sched handles the cloud VM life cycle. The idea is to simply point an existing pilot factory at a cloud scheduler.
• User-paid EC2 jobs:
  • User submits jobs with pathena to ANALY_AMAZON with the flag that it is a user type job that can only be run via the user's credentials
  • User instantiates cloud resources with "personal pilot" to grab and run the jobs. User pays the bill.
  • See PersonalPandaCloud
• Dropbox Pilot stage out mover
  • Use the dropbox python sdk to move output files into the user's dropbox folder.
  • https://www.dropbox.com/developers/releases

-- DanielVanDerSter - 19-Apr-2011