Virtualization at the Linux Plumbers Conference 2012

The 2012 edition of the Linux Plumbers Conference concluded recently.  I was there, running the virtualization microconference.  The format of LPC sessions is to have discussions around current as well as future projects.  The key words are ‘discussion’ (not talks — slides are optional!) and ‘current’ and ‘future’ projects — not discussing work that’s already done; rather discussing unsolved problems or new ideas.  LPC is a great platform for getting people involved in various subsystems across the entire OS stack in one place, so any sticky problems tend to get resolved by discussing issues face-to-face.

Continue reading

Changing GNOME Default Action for Low Battery

The GNOME default of ‘hibernate’ or suspend-to-disk on very low battery power isn’t optimal for many laptops — hibernate is known to be broken on several hardware setups, it frequently results in file system corruption, and just causes pain.  That, combined with the weird behaviour of the GNOME power manager to put the system in hibernate, even when the battery isn’t low, annoyed me enough to go hunting for a way to change the default.

The GUI doesn’t expose a ‘sleep’ setting; it just offers hibernate and shutdown, so here’s a tip to just put the system to sleep state (suspend to RAM), which is a much well-behaved default for me.

Continue reading

Workaround for error after upgrading VM from F16 to F17

Updating a Fedora 16 guest to a Fedora 17 guest via preupgrade gave me the ‘Oh no, something has gone wrong!’ screen at the GDM login screen.  It’s quite frustrating to see that screen because you can’t switch to a virtual terminal for troubleshooting, or even reboot or shutdown.

To send the key sequence Ctrl+Alt+F2 to the guest to switch to a virtual terminal, use the qemu monitor by pressing

 Ctrl+Alt+2

and use sendkey to send the key sequence:

(qemu) sendkey ctrl-alt-f2

Then go back to the guest window by issuing

Ctrl+Alt+1

After logging in as root, I poked in the gdm log files in /var/log/gdm/ and saw the fprint daemon was causing some errors.  Removing the fprintd package fixed this, but this is just a workaround, not a solution:

yum remove fprintd

Bug filed.

Using adb To Copy Files To / From Your Android Device

Some devices, like the Galaxy Nexus and the HP Touchpad* (via the custom Android ROMs) don’t expose themselves as USB storage devices.  They instead use MTP or PTP to transfer media files (limiting to only photos and audio/video files being shown from the device).

This happens due to there being no separate sdcard on these devices, and ‘unplugging’ an sdcard from a running device to be exposed to the connected computer could cause running apps on the device itself to malfunction.  Android developer Dan Morill explains this here. He also mentions how the Nexus S doesn’t have this problem.

There are several apps that can open shares to the device using one of several protocols (DAV, SMB, etc.).  However, one quick way I’ve found to copy files to and from the device connected via USB to a computer is by using the adb tool.  It’s available as part of the ‘android-tools’ package on Fedora.

To copy a file from the computer to an android device connected via usb, use this:

adb push /path/to/local/file /mnt/sdcard/path/to/file

This will copy the local file to the device in the specified location.  Directories can be created on the device via the shell:

adb shell

and using the usual shell commands to navigate around and create directories.

* On the Touchpad, WebOS can expose the storage as a USB Storage Media.  The current nightly builds of CM9 can’t.

Blog moved to wordpress on openshift

I moved this blog a while back from Blogger to WordPress. I was looking to move away from Blogger/Blogspot, to something self-hosted. I had come up with the following list to make the move seamless (for me as well as regular visitors):

  • Ability to use custom domains: Since I used blogger’s custom domains feature to redirect the blogger/blogspot links to my domain, I wanted to retain that functionality
  • Make the move seamless to site visitors
  • Preserve links and link structure.  All earlier links, rss feeds, etc., should continue to work as they did with the earlier setup (helps in maintaining search engine rankings)
  • No dependence on 3rd party server/software for leaving comments: Some blogging platforms are simple and minimal; they however end up using other services for comments to blog posts. I didn’t want that to happen — all the content should be on one sever without the users needing any sort of registration elsewhere.
  • Easy to manage the software: Shouldn’t be too time-consuming to keep the blog up

Red Hat‘s OpenShift PaaS platform had just announced support for domain aliases for applications, so I started looking at what would be involved in moving the blog on their platform.

Read on for my experiences and details on deploying this WordPress blog on OpenShift.

I already had played with OpenShift a bit, and loved their workflow of deploying apps using git. Deploying a wordpress install on OpenShift would mean I wouldn’t have to manage my own servers, operating systems, software updates, etc. It’s all on the stable and secure RHEL platform, with PHP managed by the RHEL team. So all I would need to worry about is just the wordpress installation itself.  As long as I routinely check for security updates to wordpress, and push those updates to the site, I should be doing OK.

So I created a new php-5.3 app using ‘rhc-create-app’. mysql is needed for the database, so I also added an instance to the app with the command

 rhc-ctl-app -e add-mysql-5.1 -a <appname>

To manage the mysql instance, a phpmyadmin cartridge is desirable too:

rhc-ctl-app -e add-phpmyadmin-3.4 -a <appname>

To make sure my custom domain works, let’s add aliases as well:

rhc-ctl-app -c add-alias --alias log.amitshah.net
rhc-ctl-app -c add-alias --alias www.amitshah.net

I had used both, log. and www. for the blog, so let’s continue using both so that both domains continue working. Of course, I changed the DNS CNAME entries for www. and log. over to <appname>-<domainname>.rhcloud.com via my name provider’s site.

Next, using the admin credentials on the mysql db, I then created a new db and a new user and gave the user all permissions on that db.  All this is quite simple using the phpmyadmin interface.

That’s it, all set with the app on OpenShift.

I then went and downloaded the latest wordpress release (3.2.1 then) zip file and extracted the files in a local directory.

Now here’s where I started using the power of git and OpenShift: I created a git repo in the wordpress directory and added all files to it, and made an initial commit. This is my base from where I’ll use wordpress.  New wordpress releases can be copied in this directory, and a new commit will map to the upstream release version. Any modifications to files I make in my wordpress installation (e.g. theme changes) are tracked in another branch in the same directory, with that branch being rebased on top of the latest release (the master branch).

With this setup, I can just copy the contents of this directory into my app’s php directory and push the changes to OpenShift. The ‘php’ directory is where all the app code resides. I then added all files in the git repo and committed the result. I then created the wp-config.php file as a copy of the wp-config-sample.php file, modified it to suit my installation, committed the change, and also added the file to the other wordpress directory created in the first step above. I then just pushed the changes, and the app was  live on the cloud and I could get started with wordpress’s wizard-based installation.

Now here’s one oddity of hosting apps on OpenShift: the app directory isn’t writable, or isn’t the place where the app itself can make changes and assume they’d be preserved (I think this is a good thing). Since the app is deployed via git, any content written to the server app directory can be lost on the next git push. For wordpress, this means the ‘uploads’ directory has to be given a place where images, etc., can be uploaded without problems.

The OpenShift people have helpfully given us some environment variables and hooks in the app deployment process, which can be used to do this right.

The default wordpress uploads directory is ‘wp-content/uploads’.  We can continue using this directory, with the following snippet placed in ‘.openshift/action_hooks/build’:

cd $app_dir
cat >> .openshift/action_hooks/build
if [ ! -d $OPENSHIFT_DATA_DIR/uploads ]; then
    mkdir $OPENSHIFT_DATA_DIR/uploads
fi

ln -sf $OPENSHIFT_DATA_DIR/uploads $OPENSHIFT_REPO_DIR/php/wp-content/

This ensures the ‘wp-content/uploads’ location is available for wordpress to put stuff into, and it also ensures the content goes into a place where OpenShift will not destroy the data on the next git push.

OK, having done all this, I was now ready to import my older blog posts. I installed the blogger-to-wordpress and livejournal-to-wordpress plugins (well, since I’m doing this, I thought I might as well import my older lj entries), git push’ed them, and did the import from the web interface.

Comments from livejournal entries and some blogger posts didn’t get fetched. I don’t know why that happened. I tried the import a couple more times, but those posts didn’t show up. I just decided to not bother about that; if there was any frequently-visited post, I could always go back and import it by hand. Since I didn’t expect to do any more imports, I removed those plugins and pushed the result again.

There is a blogger-to-wordpress redirect plugin, but that plugin does a lot more than just redirecting: it imports images uploaded to blogger or picasaweb on the blogger posts, generates blogger template to redirect blogger posts to wordpress, maps blogger posts to wordpress posts, etc.  Now most of this functionality is one-time; importing pictures, generating blogger template for redirection, etc., doesn’t need to be present all the time (can’t be too careful with php apps and security). I used the plugin to import all the blogger/picasaweb pictures it could fetch, and removed it as well.

I then enabled wordpress’s custom URL structure, which allows blogger-like post URLs, with the year and month as well as post title in the URL. Enabling this needs .htaccess modifications, which wordpress can’t make directly in our setup (because it can’t write to the app directory).  So created a new .htaccess file in the php/ dir. in the OpenShift app directory and included the snippet wordpress helpfully tells you it would include if the directory were writable (my code is in the snippet below).

I also took some hints from the blogger-to-wordpress plugin and created a minimal plugin that maps blogger URLs to wordpress URLs, and installed this plugin.

Next up was to ensure the older feeds kept working, and also ensuring the contents of the wp config file, and directory listings weren’t displayed. I also searched for some wordpress hardening tips, and compiled a fun-looking .htaccess file, snippet included below:

# Disable directory listing
Options All -Indexes

<files .htaccess>
    order allow,deny
    deny from all
</files>

<files wp-config.php>
    order allow,deny
    deny from all
</files>

RewriteEngine On
RewriteBase /

# Most of following comes from
# http://bloggertowp.org/migrate-from-blogger-to-wordpress-best-tutorial/

# Redirect feeds from labels
RewriteRule feeds/posts/default/-/(.*) category/$1/feed/ [L,R=301]

# Redirect older blogger RSS feeds
RewriteRule rss.xml feed/ [L,R=301]
RewriteCond %{QUERY_STRING} ^alt=rss$
RewriteRule feeds/posts/default feed/? [L,R=301]

# Redirect older blogger ATOM feeds
RewriteRule atom.xml feed/atom/ [L,R=301]
RewriteRule feeds/posts/default feed/atom/ [L,R=301]

# Redirect older blogger comments feeds
RewriteRule feeds/comments/default comments/feed/ [L,R=301]

# Redirect archives
RewriteRule ^([0-9]{4})_([0-9]{1,2})_([0-9]{1,2})_archive\.html$ $1/$2 [L,R=301]

# Redirect labels
RewriteRule ^search/label/(.*)$ category/$1/ [L,R=301]

# This is WP default: makes pretty URLs possible.
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]

I also installed the WP-Piwik and smart-404 plugins. WP-Piwik is a plugin that adds Piwik javascript code to give me a summary of the visits to the site, and the search keywords people use to land on my site. More on Piwik and its setup in a follow-up blog post. Smart-404 shows a list of pages with similar titles to the one being used in the 404 page. I had noticed a few 404 page hits via Piwik.

I’ve enabled the Akismet plugin that comes with the wordpress distribution, and it has flagged over 600 comments as spam so far, with just 2 false-positives. That’s impressive, but I intend to look further into this:

  1. Is there a way to reduce spam comments?
  2. Why do wordpress sites get spammed so much?

What I’ve seen so far is people search for specific terms on the ‘net, land on some post, and put the spam comment. So these are actual humans, not bots. Since they’re investing enough effort into finding blogs and adding comments, spam prevention techniques like CAPTCHAs aren’t going to work all the time. Akismet is working fine so far, so I’ll continue using it, but I’m going to think / search for ways to mitigate spam.

Overall, the move was really painless, done within a weekend and the most time was spent in learning about WordPress and moving the existing posts to the new blog. There were hardly any OpenShift issues, it stayed nicely out of the way, and I really like that about the platform.

I still haven’t figured out a way to map Blogger labels to WordPress Categories/Tags; these are new concepts (to me), and I’ll probably get something done here with some more htaccess trickery.

My opensource.com Article on FUDCon Pune 2011

My article on FUDCon Pune 2011 appeared on opensource.com last week:

http://opensource.com/life/11/12/fudcon-pune-making-conference

Apparently my initial submission was about 3x longer than the average article on opensource.com.  I’ve covered events running up to the conference on this blog, and with the osdc article, I’ve covered the conf as well.  There still might be a few things left which I’ll post about here in the coming days.

FUDCon Pune: My talk on ‘Linux Virtualization’

My second talk at FUDCon Pune was on Virtualization (slides) on day 2.  While I had registered the talk well in advance, I wasn’t quite sure what really to talk about: should I talk about the basics of virtualization?  Should I talk about what’s latest (coming up in Fedora 16)?  Should I talk about how KVM works in detail?  My first talk on git had gone well, and as expected for this FUDCon, majority of the participants were students.  Expecting a similar student-heavy audience for the 2nd talk as well, I decided on discussing the basics of the Linux Virt Stack.  Kashyap had a session lined up after me on libvirt, so I thought I could give an overview of virt-manager, libvirt, QEMU and Linux (KVM).

And since my registered talk title was ‘Latest in Linux Virtualization’, I did leave a few slides on upcoming enhancements in Fedora 16 (mostly concentrating on the QEMU side of things) at the end of the slide deck, to cover those things if I had time left.

As with the previous git talk, I didn’t get around to making the slides and deciding on the flow of the talk till the night before the day of the talk, and that left me with much less sleep than normal.  The video for the talk is available online; I haven’t seen it myself, but if you do, you’ll find I was almost sleep-talking through the session.

To make it interactive as well as keep me awake, I asked the audience to stop me and ask questions any time during the talk.  What was funny about that was the talk was also being live streamed, and the audio signal for the live streaming was carried via one mic and the audio stream for the audience as well as the recorded talk was on a different mic.  So even though the audience questions were taken on the audience mic, I had to repeat the questions for the people who were catching the talk live.

I got some feedback later from a few people — I missed to introduce myself, and I should have put some performance graphs in the slides, as almost all users would be interested in KVM performance vs other hypervisors.  Both good points.  The performance slides I hadn’t thought about earlier, I’ll try to incorporate some such graphs in future presentations.  Interestingly, I hadn’t also thought of introducing myself.  Previously, I was used to someone else introducing me and then me picking up from there.  At the FUDCon, we (the organisers) missed on getting speaker bios, and didn’t have volunteers introduce each speaker before their sessions.  So no matter which way I look at it, I take the blame as speaker and organiser for not having done this.

There was some time before my session to start and there were a few people in the auditorium (the room where the talk was to be held), so Kashyap thought of playing some Fedora / FOSS / Red Hat videos.  (People generally like the Truth Happens video, and that one was played as well.)  These, and many more are available on the Red Hat Videos channel on YouTube. There was also some time between my session and Kashyap’s (to allow for people to move around, take a break, etc.), so we played the F16 release video that Jared gave us.

Overall, I think the talk went quite well (though I may have just dreamed that).  I tried to stay awake for Kashyap’s session on libvirt to answer any questions directed my way; I know I did answer a couple of them, so I must have managed to stay up.

FUDCon Pune: My session on ‘Learning Git’

My session on learning git (slides) was scheduled right after the lunch break on the first day of the FUDCon Pune 2011.

I had targeted the session for beginners; however I had some help from Shakthi, who conducted a session on git during the 2nd FAD and from Ramky who spoke on version control systems in the talk before mine.  So I could skip a few basic things and get right on to the demo.

I didnt really get the luxury to prepare well in advance; I had in my mind what I would do in general, but the got the slides and the flow ready just the night prior to the talk.  Organising FUDCon wasn’t too taxing, but there are a few last-minute things that have to be done, well, at the last minute.  And the presentation, etc., had to wait.

I have earlier seen students just attend sessions but not really follow up on what they were being taught.  So I thought I’d make this an interactive session, inviting people from the audience to participate in the session by someone coming up on the stage and writing a .c program, someone else coming up and creating a git repo, then someone else modifying the code, doing another commit, and so on.

While I thought about this, I recalled Rusty’s session at foss.in a few years back where he did such a thing successfully.  Now emulating that feat would be really difficult.  People who have attended Rusty’s talks would know what I mean.  He puts in hours and days for such talks.  I’m sure he’d have thought about how to pull it off even if the person to come up on stage wouldn’t know how to type.

There were about 50 – 60 people attending the  talk.  So what I did, instead, was to ask the attendees about who knew how to write C programs, and who knew how to type fast.  I called up one such attendee and asked him to write a simple ‘Hello, World!’ program.

I then called up someone else (Aditya) to commit the first version.  Thankfully, the original C file did not have any punctuation in the ‘Hello, World!’ string, so the idea for the 2nd commit was ready.  Once Aditya initialised the git repo and did the first commit, I modified the program output to add the comma and exclamation point and make that the 2nd commit in the git repo.  I then moved on to create a new C program that prints out ‘Goodbye, World’ (we had dedicated the conference to Dennis Ritchie).  This was done in a new branch called ‘goodbye’.  Next was to create another branch, called ‘fudcon’, and write another C program to show ‘Hello, FUDCon’.  Then a few lessons on merging, switching branches, viewing commits and logs from other branches followed.  The slides have the list of commands that were shown.

The last step was to clone this repo into another local one, commit a few things there, do a push into the original repo, make some other pulls here and there, and the session participants were ready with hands-on git lessons that they could use.

I had quite a few questions during and after the session, and I even heard of people trying out the examples after the talk. So I’d call the talk/demo a success.

FUDCon Pune Day -1

Things had been going on smoothly so far: banners, posters and booklets had arrived, wireless routers had arrived, the guys in charge of adding power outlets to the venue were going to arrive on time.

Jared Smith, Joerg Simon and Robert Scheck were already in Pune that day and we hosted them at the Red Hat office after lunch.

However, with just one day left for the conference, a little amount of panic set in.

We had planned to go to COEP and register the volunteers to minimise the rush on the registration counters on the first day, and also to have a brief chat with the volunteers on how the event would proceed.  However, we were informed the students had classes till 1730 that day.  Also, the t-shirts would have to be kept on the other side of the campus (across the road) on the 3rd floor of the building (which doesn’t have an elevator).  Lugging the huge T-shirt boxes isn’t something we wanted to burden anyone with, so we decided not to do the distribution / registration that day, but call the students earlier on the 1st day.  We decided on meeting at 0730 hours to register the volunteers on day 1.  Satya, who was in contact with the volunteers, conveyed all these messages across.

However, I sensed a bit of agitation in Satya’s voice.  Looks like she had been in touch with all the speakers landing in, and they were landing all through the night.  She was then also co-ordinating with the cabs to get them to Pune (for those coming in from Mumbai).  Some speakers missed their flights, some chose to hang around in Mumbai and shop around.  Getting in touch with these folks became difficult, and with the cab drivers asking for details from Satya, it was clear to see what she had gone through: no sleep, and working overtime to figure out if everyone is safe and sound and coming in properly.

She wasn’t complaining, but definitely there was something wrong.  She had too much to do on her plate.  I’m not sure how that happened; we had delegated most of the stuff to people, but Satya somehow got caught up with volunteers, registration desk and hospitality.  And all of these things got pretty active in the final days.  We’ll have to keep that in mind for the next time.  Satya’s been a great sport, though, and she’s ensured people got in and out of Pune on time.

A note to speakers and those whom we approved travel sponsorship for: please help us help you.  Keep a phone handy and call us immediately in case of any change of plans (or delays).  Please respond to all emails we send out asking for information and call us the moment something unexpected happens.  Helps maintain everyone’s sanity!

Back to the action: while Prasad (PJP), Shreyank and Kashyap went ahead to the venue to set up banners, power outlets and wireless routers, the rest of us stayed back at the office, started the registration session for Red Hat speakers and attendees, handing out their badges.  Things weren’t too smooth for the folks who went to the venue, either.  Power had failed and they couldn’t test the wireless capabilities.  The electricians adding power outlets could go on doing their work using mobile phone flash lights, though.  It took a good hour and a half for the power to be restored.  We worried if the event could go on without glitches…

Quotable Quotes and Videos from FUDCon Pune 2011

Jared Smith: ‘Our biggest enemy is ourselves’

Sitaram Chamarty: ‘English is never going to be as precise as perl or shell. That’s my biggest problem in life.’

Joerg Simon (kital): ‘The event was really smooth.  All talks started on time, with German precision.  Almost feels like a German event.’  (In an offline chat with me).

Jared Smith: ‘Best FUDPub EVER!’ (on-video, off-video, in multiple locations, in multiple voices)

The kpoint folks have the event videos at their site.

Amit Shah's blog