ydal

tl;dr ver­sion: If you can’t use Apps2SD, do adb shell, pm setInstallLocation 2, move any app to SD (igno­ring pos­si­ble “fai­led” errors at first try).

Our beloved Fro­zen Yoghurt came with many new fea­tures wel­come to the com­mu­nity at large, and one fea­ture which had a mixed recep­tion: “Apps on exter­nal sto­rage”, which allows the user to install app­li­ca­ti­ons to its phone’s exter­nal sto­rage — mostly in order to free up inter­nal disk space.

Many custom ROM dis­tri­bu­ti­ons for Android alre­ady had this fea­ture built in, going by the moni­ker “Apps to SD” (or “Apps2SD” or just “A2SD”).

The typi­cal imple­men­ta­tion of A2SD works by using an ext2/ext3 par­ti­tion on the SD card of your device — and usually only works when it’s exactly the second par­ti­tion. For the sake of argu­ment, one such custom imple­men­ta­tion of A2SD will be inclu­ded at the end of this post.

What it then does is just com­ple­tely move all the app­li­ca­ti­ons to the SD par­ti­tion, lea­ving only the /data par­ti­tion behind, and uses a bind mount to fool the sys­tem into belie­ving that the files are still on the same file sys­tem. So, in essence, the a2sd patch “cheats” and pre­tends that not­hing actually has hap­pened while quietly sipho­n­ing the apps to the SD card.

This, of course, only works when you actually have root access to your device and are allo­wed to play around with all the inte­res­ting sys­tem data its­elf. If you’re working on an unrooted/stock hand­set and firm­ware, you don’t have the option of using this fea­ture; and also if you’re too lazy or unkno­wing or prissy to set up an ext[23] par­ti­tion on your SD card.

Thus the «offi­cial» Apps to SD comes into play — if your device is run­ning Android 2.2, that is.

An import­ant thing to note about under­stan­ding the offi­cial imple­men­ta­tion is that it assu­mes that the user has no direct access to the /system par­ti­tion. Espe­cially: the user is not able to access any instal­led Android app­li­ca­tion package in any way that allows copy­ing files.

What Froyo does when instal­ling an app­li­ca­tion to SD is pretty sim­ple: it crea­tes a file on the SD card and uses this as a con­tai­ner to store the app­li­ca­tion in. Said con­tai­ner is used with a cryp­ted loop mount, that is the actual data on the SD card is encryp­ted, and will be decryp­ted at load time when acces­sing the application.

The idea behind this see­min­gly con­vo­lu­ted setup is sim­ple: if you have paid for an app­li­ca­tion, you could just store it on SD and then copy it if it is not encryp­ted. If it is encryp­ted, you can­not access the app­li­ca­tion in a “sim­ple” way to copy (i.e. pirate) it.

Addi­tio­nally, the app­li­ca­tion (with the default set­tings) needs to allow Android to move it to the SD card — other­wise the sys­tem does not enable the func­tio­na­lity, pro­bably to ensure that app­li­ca­ti­ons aren’t “bro­ken” by SD storage.

Of course this is easily man­hand­led by using the USB debug­ging inter­face with adb shell: just issue pm setInstallLocation 2. This tells he package mana­ger (hence «pm») to use the exter­nal sto­rage as a default install loca­tion, which inci­den­tally lifts the block that does not allow an app­li­ca­tion to be stored on exter­nal sto­rage, too.

The down­side:
/dev/block/dm-41 on /mnt/asec/de.hafas.android.db-1 type vfat [...]

And yes, that’s 41 device map­per crypto loops. At least they don’t pro­duce that much over­head as to noti­ce­ably slow down the system.

One of the boons of the Froyo imple­men­ta­tion is that with above com­mand, it can easily be used even with an unroo­ted phone and wit­hout repar­ti­tio­ning your SD drive. The disad­van­ta­ges are that Android requi­res a fair bit of time after boo­ting to mount all the crypto loop devices, which will result in your app­li­ca­ti­ons being acces­si­ble rather late after boo­ting. Also, you will not be able to use wid­gets of any app that is on SD.

Here come the advan­ta­ges of the cust­o­mi­zed A2SD approach: you can still access wid­gets and app­li­ca­ti­ons on your SD card even when it is moun­ted to your com­pu­ter — because Android will only mount away the root par­ti­tion (the FAT one), and not your ext par­ti­tion. And you’ll have less over­head due to the crypto business.

And, as pro­mi­sed, the code that enab­les A2SD on most cur­rent ROMs:

#!/system/bin/sh
#
# Apps2SD using symlinks and bind mounts
# Originally by cyanogen (shade@chemlab.org)
# Modified to use a cleaner /sd-ext implementation by IEF (ief@shadowchild.nl)

# execute any postinstall script then kill it
if [ -e /dev/block/mmcblk0p2 ];
then

    # mount and set perms
    busybox mkdir /sd-ext
    busybox mount -o noatime,nodiratime -t auto /dev/block/mmcblk0p2 /sd-ext;
    busybox chown 1000:1000 /sd-ext;
    busybox chmod 771 /sd-ext;

    # clean up any old symlinks, create data directories
    for i in data;
        do
                if [ -h /data/$i ];
                then
                        rm /data/$i;
                fi;
                if [ ! -d /data/$i ];
                then
                        mkdir /data/$i;
                        busybox chown 1000:1000 /data/$i;
                        busybox chmod 771 /data/$i;
                fi;
        done;

    # don't allow /data/data on sd because of upgrade issues - move it if possible
    if [ -d /sd-ext/data ];
    then
        busybox cp -a /sd-ext/data/* /data/data/;
        busybox rm -rf /sd-ext/data;
    fi;

    # move apps from internal memory to sdcard
    for i in app app-private dalvik-cache;
    do
        if [ ! -d /sd-ext/$i ];
        then
            mkdir /sd-ext/$i;
        fi

        busybox chown 1000:1000 /sd-ext/$i;
        busybox chmod 771 /sd-ext/$i

        if [ -d /data/$i ] && [ ! -h /data/$i ];
        then
            busybox cp -a /data/$i/* /sd-ext/$i/;
            busybox rm -f /data/$i/*;
        fi;
    done;

    # symlink app dirs - they must be on the same filesystem
    for i in app app-private dalvik-cache;
    do
        if [ -d /data/$i ] && [ ! -h /data/$i ];
        then
            busybox rm -rf /data/$i;
            busybox ln -s /sd-ext/$i /data/$i;
        fi;
    done;

    # clean up old whiteouts
    for i in local misc property system tombstones data;
    do
        if [ -h /sd-ext/$i ]; then rm -f /sd-ext/$i; fi
    done;

    # please don't put odex files in the app directory people!
    # it causes dexopt to crash when switching builds!
    busybox rm -f /sd-ext/app/*.odex

    setprop shadow.apps2sd.active 1;

    echo "+++ Apps-to-SD successfully enabled";

else

    # replace symlinks with directories so we can boot without sd
    for i in app app-private dalvik-cache;
    do
       if [ -h /data/$i ];
       then
            rm -f /data/$i;
            mkdir /data/$i;
            busybox chown 1000:1000 /data/$i;
            busybox chmod 771 /data/$i;
        fi;
    done;

    setprop shadow.apps2sd.active 0;
fi;
sync;

This is run as an init script.

A cursory glance at how different «Apps to SD» implementations work on Android.

I recently stum­bled upon the rather neat vimium exten­sion for Chrom(e|ium), which does much the same as the vim­pe­ra­tor exten­sion for Fire­fox. The pro­blem, though, as with vim­pe­ra­tor and vim its­elf, is that the default key­board map­pings are a bit of a pain in the arse for Dvorak users, as hjkl isn’t on the home row any­more, much less next to each other.

The­re­fore, it needs some remap­ping to get in a half­way fami­liar and Dvorak-compatible lay­out, which would look like this:

unmapAll

map r reload
map e removeTab
map u restoreTab
map h scrollDown
map t scrollUp
map d scrollLeft
map n scrollRight
map <c-h> scrollPageDown
map <c-t> scrollPageUp
map <c-u> scrollFullPageDown
map D goBack
map N goForward
map T nextTab
map H previousTab
map <c-y> createTab
map gg scrollToTop
map G scrollToBottom
map gf toggleViewSource
map zi zoomIn
map zo zoomOut
map yy copyCurrentUrl
map i enterInsertMode
map f activateLinkHintsMode
map F activateLinkHintsModeToOpenInNewTab
map / enterFindMode
map . performFind
map , performBackwardsFind

Just paste it in the remap field of the extension’s “advan­ced opti­ons” menu.

Dvorak keyboard mappings for the vimium extension.

As pisg is ill-equipped to handle sup­port for ‘admin’ users in the stan­dard con­fi­gu­ra­tion, I went on a quick code hunt to find the bit of code responsi­ble for strip­ping nick modes from a log line. A bit counter-intuitively, this func­tion is cal­led normalline, and not some­thing like normalize or strip_mode.

Anyhow, here’s a small patch to fix the pro­blem for the Irssi par­ser module:

--- modules/Pisg/Parser/Format/irssi.pm.old	2008-02-13 21:40:25.000000000 +0100
+++ modules/Pisg/Parser/Format/irssi.pm	2010-03-16 02:29:42.000000000 +0100
@@ -10,7 +10,7 @@
     my ($type, %args) = @_;
     my $self = {
         cfg => $args{cfg},
-        normalline => '^(\d+):\d+[^<*^!]+<[@%+~& ]?([^>]+)> (.*)',
+        normalline => '^(\d+):\d+[^<*^!]+<[@%+~&! ]?([^>]+)> (.*)',
         actionline => '^(\d+):\d+[^ ]+ +\* (\S+) (.*)',
         thirdline  => '^(\d+):(\d+)[^-]+-\!- (\S+) (\S+) (\S+) (\S+) (\S+)(.*)',
     };

Or you could just down­load the diff directly.

Instructions on patching support for the euIRC "!" user mode into pisg.

I was brow­sing through some older files of mine and clea­ning up when I stum­bled over a bunch of old instant mes­sen­ging log files. These logs were still in the legacy uni­fied log file for­mat which Gaim (today knows as Pid­gin) used in the begin­ning. I didn’t find a con­ver­ter after about ten seconds of using Google, so I went ahead and wrote my own.

It’s mostly feature-complete, and will split up any num­ber of [something].log you pass to it into [something]/[date].txt style files. What it can’t really do is deter­mine what kind of pro­to­col it’s dea­ling with, so you’ll still have to move the log direc­tory manu­ally to the appro­priate pro­to­col direc­tory inside of ~/.purple/logs. Be wary when moving files, though, as you might acci­den­tally over­write other log files. Use rsync.

Anyhoo, you can eit­her get the file directly or just try this deli­cious copypasta:

#!/usr/bin/perl
# gaim2pidgin.pl
# author:  towo <towo@ydal.de>
# version: 3
# license: CC-BY-DE-3.0
 
use strict;
 
# convert short month names to numbers.
my %shortmonths = (
	'Jan' => '01',
	'Feb' => '02',
	'Mar' => '03',
	'Apr' => '04',
	'May' => '05',
	'Jun' => '06',
	'Jul' => '07',
	'Aug' => '08',
	'Sep' => '09',
	'Oct' => '10',
	'Nov' => '11',
	'Dec' => '12'
);
 
# go through files
FILE: foreach my $file (@ARGV) {
	my ($header, $target);
 
	# sanity checks
	unless (-f $file) {
		warn "$file is not a file.\n";
		next FILE;
	}
	unless(open(LOG, $file)) {
		warn "Unable to open $file for reading: $!\n";
		next FILE;
	}
 
	# get file header, get target name
	chomp($header = <LOG>);
	$header =~ s#<.*?>##g;
	$target = $file;
	$target =~ s/\.log$//;
 
	# check header for correctness
	unless($header =~ m{^(<HTML><HEAD><TITLE>)?IM Sessions with .*? \
                             (</TITLE></HEAD><BODY BGCOLOR=".*?">)?$}i) {
		warn "$file does not seem to be a gaim conversation.\n";
		next FILE;
	}
 
	# read LOG to file
	my @contents = <LOG>;
	close(LOG);
 
	# parse log file (one loop ^= one chat session)
	while(@contents) {
		my ($session, $identifier, $date);
 
		# get session identifier
		chomp($session = shift @contents);
 
		# Strip HTML.
		#$session =~ s#<.*?>##g;
		$session =~ s#</?(FONT|B|I|ALIGN|HTML|HEAD|TITLE|HR|BR|BODY|H3).*?>##ig;
 
		# sanity check for the session identifier
		unless ($session =~ m/^ ?---- New Conversation @ \w{3} (\w{3}) ([0-9 ]{2}) \
                                      (\d{2}):(\d{2}):(\d{2}) (\d{4}) ----$/) {
			warn "Could not recognize session identifier: «$session»\n";
			next FILE;
		}
 
		# extract date from session identifier and create target identifier
		$date = "$6-$shortmonths{$1}-" . sprintf("%02d", $2) . ".$3$4$5";
		$identifier = "$target/$date.txt";
 
		# sanity check for target directory
		unless (-d $target) {
			unless(mkdir $target) {
				warn "Could not create directory $target: $!\n";
				next FILE;
			}
		}
 
		# open output file
		unless(open(OUTPUT, "> $identifier")) {
			warn "Could not write to $identifier: $!\n";
			next FILE;
		}
		select OUTPUT;
 
		# extract log to log file
		until($contents[0] =~ m/^(<HR><BR><H3 Align=Center>)? ?---- New Conversation/ \
                                      or !@contents) {
			my $line = shift @contents;
			$line =~ s#<.*?>##g;
			print $line;
		}
		close OUTPUT;
	}
}

Licen­sed as CC-BY-DE-3.0.

Simple converter of old flat file gaim logs to Pidgin directory logs.