CometCloud is an autonomic computing engine for Cloud and Grid environments. It is based on the Comet decentralized coordination substrate, and supports highly heterogeneous and dynamic cloud/Grid infrastructures, integration of public/private clouds and autonomic cloudbursts. CometCloud provides a shared coordination space over the Chord overlay network and various types of programming paradigms such as Master/Worker, Workflow, and MapReduce/Hadoop.

CometCloud supports autonomic cloudbursts and autonomic cloud-bridging on a virtual cloud which integrates local computational environments and public cloud services on-the-fly. Also it supports real-world scientific and engineering applications.

What is CometCloud capable of?

  • Support core programming paradigms for real-world data and compute intensive applications
  • Enables autonomic cloudbursts and cloud-bridging and on-demand scale-out and scale-in, driven by dynamic policies, economic model, QoS constraints, etc.
  • Programming system support deployment of native (Java as well as non-Java) applications without any change.
  • Current deployments include a virtual cloud integrating Amazon EC2, Eucalyptus, local clusters, and TeraGrid.
  • Incorporates mechanisms for fault-tolerance to handle node and communication failures and recovery.

 

What are CometCloud Features?

  • Pull-based task consumption: workers pull tasks from the Comet space whenever they are available
  • Policy-based autonomic cloudbursts: CometCloud supports on-demand scale-out and scale-in by policy and economic model.
  • Cloud-bridging: CometCloud provides a cloud-bridging over a virtual cloud which integrates datacenter, clouds such as Amazon EC2 and Eucalyptus, and Grids such as Teragrid.
  • Support for multi-core processors: users can run multiple workers on a node and utilized multiple cores.
  • Support MapReduce with naïve disk writing: instead of using Hadoop file system, CometCloud uses the Comet space for storing data to support MapReduce and writes data into the disk if available memory amount goes below a threshold.
  • Master throttling: CometCloud can control the rate of task generation of masters. If the rate is high, then more tasks keep in the space.
  • Multiple masters: multiple masters can generate tasks and insert them into the same Comet space to collaborate.
  • Task update: if a task with the same task id is inserted, the existing task is updated.
  • Garbage collection: if tasks remain in the space for some time without being consumed, then they are cleared.
  • Distributed task generation by workers: even workers can generate tasks and insert them into the space. Because every task should have a unique task id, CometCloud provides a way to provide a globally unique id to distributed workers.
  • Fault-tolerance by replicating task space: each node replicates the preceding node’s local space, and it merges the replicated space when its preceding node fails.