Code, Bass, and Rainy Weather: November 2011

Wednesday, November 9, 2011

Dorrie

So after abandoning my misadventures of last week, I decided to clone the Dorrie project, and use that as the underlying base for the remixerator. The bread and butter of this project is in the pykickstart.parser module, which has many tools to parsing and generating a working kickstart file. I will briefly explain the key functions and objects that are used in dorrie.

pykickstart.parser defines three classes:
KickstartParser - a state machine used when going through a kickstart
Packages - an object that encapsulates all the selected (and removed) packages of the kickstart.
Script - an object that encapsulates a script of the kickstart file.

The KickstartParser's most useful function for us is the readKickstart() function. This, given a kickstart file, will go through (using the state machine internally) and create all the objects necessary to represent a particular kickstart. This uses internal parsing and handling functions. It also has an internal handler object that saves and executes commands that are parsed.

The Packages object has four important lists. The lists excludedList and packageList, that contain the actual packages for the kickstart (the former being denoted with a '-' character). The lists groupList and excludedGroupList are also included in this module. Essentially, when we want to add or remove any particular package, we add the package name to the respective lists.

The Script module has two important fields: script and type. Each script object should encapsulate only one script, the script field set to the actual script's string, and the type set to the relevant type found in pykickstart.constants. Creating a script object is not difficult, however, adding it in, is. Pykickstart has a BaseHandler object (to which KickstartParser's handler object is a subclass), that contains a scripts section, holding these instances of the script object.

One of the difficulties in using the dorrie project for the remixerator is that adding a post script is non-trivial. The documentation states that to add a script, you call KickstartParser.addScript... but this function actually doesn't exist. For the time being, one way to get around this is to literally append %post script to the returned formatted string. I did this for the sake of time, but this will be changed when I can find a more proper way to add a script object.

After this, I had to go through Dorrie's settings.py file and change references to reflect fedora 16. This included versions of the internal parser (these are defined in the latest pykickstart repository), and package trees.

Next is a little bit of de-chroming, to get the aesthetics looking like an RIT project.

Edit: Scripts get added when the Script "section" object has its finalize method called. I'll post more on this later.

Thursday, November 3, 2011

HTML and JQuery forms on Twisted

When I started to prototype a remixerator, I created a simple form front-end using simple html and JQuery.

Here's the head. I used this nice css theme generator to get an aesthetic that worked, put in a few of my own tweaks, and imported the necessary JQuery files. In my particular form, I liked the idea of having "sections" as tabs, and "subsections" represented by the accordion layout. When using accordions inside tabs, make sure to bring in accordions FIRST and tabs SECOND or else they won't render correctly.

<head>
<title>Remixerator Prototype</title>
<link type="text/css" href="css/custom-theme/jquery-ui-1.8.16.custom.css" rel="stylesheet" />
<link type="text/css" href="css/overall.css" rel="stylesheet"/>
<script type="text/javascript" src="js/jquery-1.6.2.min.js"></script>
<script type="text/javascript" src="js/jquery-ui-1.8.16.custom.min.js"></script>
<script type="text/javascript">
$(function(){
// Accordion
$("#accordion").accordion({ header: "h3" });
// Tabs
$('#tabs').tabs();

$("#RemixPostForm").submit(function(e){
console.log("Entering Submit");
$.post('http://localhost:8080/handler.html', function(data) {
console.log("Entering Submit Callback");
});
});
});
</script>
</head>

Now for the body. Tabs are represented as a list with ids, and each tab's content is a div with a matching id. Otherwise, this is pretty standard html form stuff. I spent [too much] of my time dealing with css issues with this form.

<body>
<form id="RemixPostForm">

<h2 class="demoHeaders">Remixerator</h2>
<div id="tabs">
<ul>
<li><a href="#tabs-wall">Wallpaper</a></li>
<li><a href="#tabs-apps">Applications</a></li>
<li><a href="#tabs-create">Create</a></li>
</ul>

<div id="tabs-wall">
<h3>First, select your wallpaper.</h3>
<div id="wallpaper_table" align = "center">
<table id="imagegrid" border="0" width="400">
<tr>
<td align="center"><img src="images/ritwallpaper.jpg" alt="gallery thumbnail" height = "80" width = "120"/></td>
<td align="center"><img src="images/f15wallpaper.png" alt="gallery thumbnail" height = "80" width = "120"/></td>
</tr>
<tr>
<td align="center"><input type="radio" name="ritwallpaper_cb" id="ritwallpaper_cb" checked="checked"/></td>
<td align="center"><input type="radio" name="f15wallpaper_cb" id="f15wallpaper_cb"/></td>
</tr>
</table>
</div>
</div>

<div id="tabs-apps">
<h3>Next, choose the applications you want.</h3>
<div id="accordion">
<div>
<h3><a href="#">Office and Text</a></h3>
<div>
<input type="checkbox" name="vim_cb" id="vim_cb"/>Vim<br>
<input type="checkbox" name="emacs_cb" id="emacs_cb"/>Emacs<br>
<input type="checkbox" name="nano_cb" id="nano_cb"/>Nano<br>
<input type="checkbox" name="libre_cb" id="libre_cb"/>Libreoffice<br>
</div>
</div>
<div>
<h3><a href="#">Video, Images, and Music</a></h3>
<div>
<input type="checkbox" name="inkscape_cb" id="inkscape_cb"/>Inkscape<br>
<input type="checkbox" name="gimp_cb" id="gimp_cb"/>Gimp<br>
<input type="checkbox" name="dia_cb" id="dia_cb"/>Dia<br>
<input type="checkbox" name="banshee_cb" id="banshee_cb"/>Banshee<br>
</div>
</div>
<div>
<h3><a href="#">Communication and Internet</a></h3>
<div>
<input type="checkbox" name="firefox_cb" id="firefox_cb"/>Firefox<br>
<input type="checkbox" name="elinks_cb" id="elinks_cb"/>ELinks<br>
<input type="checkbox" name="midori_cb" id="midori_cb"/>Midori<br>
<input type="checkbox" name="pidgin_cb" id="pidgin_cb"/>Pidgin<br>
</div>
</div>
<div>
<h3><a href="#">Utility</a></h3>
<div>
<input type="checkbox" name="zsh_cb" id="zsh_cb"/>Z-Shell<br>
<input type="checkbox" name="git_cb" id="git_cb"/>Git<br>
<input type="checkbox" name="htop_cb" id="htop_cb"/>Htop<br>
<input type="checkbox" name="powertop_cb" id="powertop_cb"/>Powertop<br>
<input type="checkbox" name="screen_cb" id="screen_cb"/>Screen<br>
<input type="checkbox" name="wine_cb" id="wine_cb"/>Wine<br>
</div>
</div>
<div>
<h3><a href="#">Fun and Games</a></h3>
<div>
<input type="checkbox" name="nethack_cb" id="nethack_cb"/>Nethack<br>
</div>
</div>
</div>
</div>
<div id="tabs-create">
<h3>Now click the button below to start making your remix.</h3>
<input type="submit" value="Go!" name="start_button" id="start_button" />
</div>
</div>
</form>
</body>

Lastly, the twisted bits. I simply created a simple service, server.py, which hosted the form, and had the handler.


from twisted.web import static, http, resource, server

from twisted.web.server import Site

from twisted.web.static import File

from twisted.internet import reactor

from twisted.web.resource import Resource

import cgi

import sys



class FormPage(Resource):

  isLeaf = True

  allowedMethods = ('GET', 'POST')

  def __init__(self):

    resource.Resource.__init__(self)

    def render_GET(self, request):

      return self.render_POST(request)

      def render_POST(self, request):

        #Modify Kickstart, do validation

        return '<html><body>ISO Created!</body></html>'



if __name__ == "__main__":

  root = static.File('/Users/eitanromanoff/Documents/github/Remixerator')

  indexPage = Resource()

  formHandler = FormPage()

  root.putChild('index.html', indexPage)

  root.putChild('handler.html', formHandler)

  reactor.listenTCP(8080, server.Site(root))

  reactor.run()

The Remixerator

I've been working on a new project as of late that uses the kickstart tinkering I did at the start of the quarter, and takes it a step further. The goal is to create a "remixerator" - an interface, ideally a web-based one, that allows a user to create a custom fedora remix (that is still RIT-themed in content and aesthetics). The end goal would be to put it into a machine similar to the kiosk, and have a very controlled and automated process for taking in a DVD, customizing the remix, and then burning the image for a user.

The customization of the remix is easy enough - it's just a generation of a kickstart. All of my custom scripting that was done can still be injected into the kickstart, what really changes is the configuration of applications. To start, I looked into various existing technologies that do similar things.

LiveUSB creator was not really relevant to what I was trying to do.
SuseStudio was very similar, but based on several cloud technologies that are not necessary to my project at hand. Furthermore, our RIT remixes were to run Fedora.
Kickstarter was too much targeted towards Meego.

And there were several other applications as well.

While I was no eager to re-invent the wheel, none of the above projects really fit the bill. Ultimately, I decided that all I would have to do is make a web form (perhaps take the opportunity to learn some JQuery), generate a kickstart on submit, run the creation process, and serve up a webpage when it was finished.

The form was easy enough, just a bunch of html and JQuery running on twisted. Hosting things on twisted was pretty simple, too, to get started. However, things quickly became less trivial. Launching the script on a post was simple, but being able to constantly update a page, given the output of the create livecd process, would be difficult to tackle, and it was hard to believe there was another tool available that did some of this for me.

Looking again the other day, I stumbled across the dorrie project, which does much of what I wanted to do. It also has a convenient spot in its parser where I could likely inject the postscript sections that I wrote for the RIT remix. I'll take this approach from here on out, and bother with aesthetics later.