I've been creating a cloud management and automation utility during the daylight hours the last couple of months. I'm using Mojolicious to manage the REST routes and UI, and some custom backend Perl apps and modules to manage the vcloud, spacewalk and other APIs.
I chose not to use any existing Vcloud perl modules as none of them were v1.5 compatible. I was seeing some slowness in the v1.0 compatibility calls, and I found that >50% of the API calls I wanted weren't coded up anyway and I was having to parse more XML than I wanted to with someone elses module that I didn't feel like modifying. VMWare, why did you stop making your SDKs for your new projects in perl? :(
I chose Mojolicious as the framework for the REST interface because the last time I did any web development in perl, CGI::Application was the tool of choice and I was automating system information gathering and IT-like functions, back in the early 00's. REST is all the rage these days and a long-polling API call with a 20 second wait in the browser without feedback just doesn't work anymore.
I looked at Dancer and Catalyst and both are similar in spirit, but Mojo just seemed more intuitive. I have a complete toolkit of everything webby that I need. I do wish some of the eccentricities were better documented, but they are easy enough to pick up, and many are just because I've never used a modern web framework before. I've had a couple of questions and the mailing list and bug tracking folk are friendly and quick to respond, another bonus.
I'm using various other tried-and-true CPAN modules for the Controller and Agent pieces, and MongoDB for the database. I had no choice in the cloud software or package management.
The REST framework authenticates a user and builds, loads, and configures apps, which consist of multiple machines running dozens of customized RPMs, each requiring dozens of steps to configure. Its a non trivial app, and the cloud management was the easy part.
Most modern Cloud Management Suites have similar designs for a reason:
- A Cloud, basically a virtual cluster with some sort of framework for creating/deleting/manipulating VMs and networks.
- A Package Manager - this could be "app libraries", or RPMs, or some other deployable code.
- A Controller, some sort of Master Planner that takes tasks and translates them into action. Some actions it will need to have might be:
- queries the APIs at certain intervals to populate a DB
- create vms with some network and some resources and put some pkg on them and configure with XML
- reset and reload a vm
- update a vm with the latest pkg
- A Database, something that can store the information about the various infrastructure and VM pieces.
- A Front end, where the end users can feed actions to the Controller
- An Agent, a Pub/Sub client commando which resides on your various VM/and maybe infrastructure pieces that collects running data and feeds back commands to the VM from the controller and logs its actions. (commands might be restart app, update app, etc)
To get off the ground quickly, I started with Mojolicious and built REST calls and used templates for GUI interaction and javascript for ajax fun to keep long page load times down on long polls to the various APIs. No database or agent, just straight queries against everything. This was a quick way to get everything off the ground quickly and define the process and I'm glad I did it this way. I'll followup with more posts on the rest of the design choices and its evolution from there.
I didn't get "No database or agent" is it possible? :S
ReplyDeleteIn a cloud, your data has to be stored somewhere - by data I mean a record of VMs and user rights and authentication services.
ReplyDeleteWe used Vmware vCloud Director as our cloud management software, Spacewalk (Redhat Satellite) for package management, and Active Directory for user authentication.
vCloud has a very rich API for accessing information about your VMs and applications. When a user needed to see what Applications they had deployed, I would query the API for a list of user owned apps. When the user needed to see what package levels were installed on a VM, I'd use the information gathered from vCloud to access the machine and grab information from Spacewalk. All of these services were tied together with Active Directory for authentication.
To say that 'I used no database' is not entirely true; instead, I leveraged the information that was already stored in *other* databases.
The REST application I created would do tiny steps, and I had a javascript page that would call the individual routes to reduce the chance of the browser looking like it was 'hung'. As vCloud only allows a single session per user, this would break if the same user tried to do more than one thing at a time.
I hope that helps explain what I meant by 'no database or agent'.