Showing posts from March, 2009

Data Migration Done Right

If you've ever had to migrate large amounts of data from one schema to another with non-trivial structural differences in entity definitions or relations then you'll know that its no simple task and the effort required should not be under-estimated.
Data migration strategies can take many forms, from simplistic manual exporting/importing (e.g. you write a few SQL expressions to dump data out to a CSV format then use a loader to import into your new schema) to more sophisticated custom migration programs (e.g. a custom JDBC Java application with logic to query data from the legacy schema and intelligently insert into the new schema).
For complex projects, the first solution is rarely feasible. Migration often requires careful consideration of how keys and relationships should be sequenced, split, massaged and re-joint as well as various complex data clensing operations. A CSV approach is often lacking in flexibility for advanced cases. The latter solution on the other hand, i.e.…

How Dropbox saved my life

Maybe that's a bit of an exaggeration. A more accurate title should read: How Dropbox saved three hours of my life...or maybe it was only one hour, I wasn't really working that hard now that I think about it...
I've been using my Dropbox target folder as a My Documents so to speak, saving everything work/study related directly under the root tree such that it gets synched and versioned as soon as I click the save button in most cases (except Visio, which places a read-lock on open files preventing Dropbox from doing its magic until you close them, grrr!).
I was working on a LaTeX document with TeXnicCenter the other day when my Lenovo X300 froze up on me without warning yet again (it does this around once every two days or so, appears to be a known problem that Lenovo's ignoring, read some of my other posts for more info if interested). The only way to regain control when this happens is to do a forced power down and reboot. 
Luckly I wasn't too worried as I save quit…

Farewell VLC Media Player

VLC Media Player has been my video playback solution of choice for the last two or three years till now. The best thing about VLC was that it came with its own codecs embedded within, which meant that you didn't need to go chasing separate codec packs every time you got a new video file and risking the chance of installing conflicting drivers. Most of the time VLC just worked out-of-the-box, flawlessly playing many files that other apps struggled with.
Recently however I've been watching some TV series encoded in the .mkv (Matroska) format. I noticed the playback was a little sluggish at first, but I thought it was either due to poor encoding or to my new laptop which is running a low-voltage 1.2GHz CPU. I've put up with it since it wasn't too bad, but recently I tried out some other .mkvs and performance was extreamly poor. I tried these same video files on my other laptop (1.8GHz) and on my desktop (3GHz) with very similar results. The only common factor across system…

LaTeX + TeXnicCenter + MikTex + JabRef

After many years of putting it off, I finally decided to give LaTeX a proper go for writing some academic articles. For those that may have heard the term but are still unclear as to what LaTeX actually is, I like to think of it as a HTML syntax for creating PDFs and professional looking documents.
In other words, LaTeX is a markup language, so when writing a LaTeX paper you have to add LaTeX tags throughout to define how the document should be rendered at a later stage, but the document itself is always written in plain-text with no formatting, no indentation, no justification, no images, etc.
As a final comparison, typical word processors such as MS Office, OpenOffice and Google Docs are what you'd call WYSIWYG (what-you-see-is-what-you-get) editors, whereas LaTeX is merely a syntax for formatting documents, not even an editor.
Thus, much like HTML, if you want to start creating documents and articles, you'll first need an editor. While Notepad and WordPad would do the job, the…

Dropbox + SVN Repository

Using a source-control system is a useful practice even on solo non-trivial projects. Even though you don't need to worry so much about the collaborative aspects, you still need to version your code so that you can roll-back in case you've done some bad refactoring or have taken a wrong design path.

Luckily, setting up a local SVN Server on a Windows machine is incredibly easy, especially if using the free VirtualSVN package. You can install, configure and connect to a repository in under a minute!
Since your SVN repository is stored on your local machine however, you still need to protect yourself against hardware malfunction or accidental data loss. Simplest way to do this is to configure the VirtualSVN server to place the repository root folder in your Dropbox target folder.
You can then use your favourite IDE with an SVN plugin to version your code, create branches, etc, whilst everything is transparently stored in your SVN and replicated to your online Dropbox account. Sweet…

Bypass installer .NET Framework Version Checks

While trying to install the Orthagonal Toolbox plugin for Visio I was presented with the error message "This setup required .net framework version 1.1.4322. Please install the .NET Framework and run the setup agian."

Strange thing is I already had .NET Framework 3.5 SP1 installed...
The easiest solution to get around the problem was to download the free Microsoft Orca msi editor (Google it if the link breaks), open up the msi installer file, navigate to Custom Actions on the left, and delete DIRCA_CheckFX and VSDCA_VsdLaunchConditions.

After this, it's just a matter of saving the msi and rerunning the installer again. This time it skips the .NET Framework version check but continues to run as all the necessary libraries are already installed.

No ERD to SQL code generation in Visio 2003/2007 Pro

I just spent a couple of days designing a complex database ERD in Visio 2003 Professional thinking that once I was done I could just click a button and have it generate all my SQL CREATE table statements and all referential rules, etc. I thought wrong.
The ERD to SQL Forward code generation feature is nowhere to be found in Visio 2003 Pro, nor is it in 2007 Pro. This seems incredibly bizzare as it's the most useful feature you could have in an ERD designer. What's the point of spending time and effort to create a complex diagram and then having to hand-write all the SQL yourself even though all the information is already modeled?
Doing a Google search looking for solutions suggests the feature is only available in Visio Enterprise Architect 2003 and Visio Architect 2005. Ok, fair enough, but, (there's always a but) you can only install VEA2003 if you already own and have installed Visual Studio 2003 Professional, likewise you need VS2005 to install VEA2005. Ok, so Microsoft …

Blogger Data API + TinyMCE + Dropbox

I've talked about how much I hate the Blogger visual post editor in some of my previous posts and won't mention the reasons why again here. Google seriously needs to put some effort into giving us a better tool, it fails miserably in comparison with WordPress and other modern Content Management Systems.
It may be possible however to create a far superior custom administrative backend for hosted Blogger accounts by taking advantage of the Blogger Data API, TinyMCE and Dropbox.
The Blogger API exposes methods for authenticating with a Blogger account and adding, updating posts, etc. Not only that, but the Blogger API is also exposed via a JavaScript library allowing for a fully client-side administrative interface.
TinyMCE is an open-source LGPL licensed visual HTML editor that puts the Google Blogger WYSIWYG editor to shame. I believe current WordPress versions use a variant of this (but I could be mistaken).
It should thus be possible to code up a simple HTML page with the neces…

Lenovo X300 still freezing

My new Lenovo X300 laptop, freshly formatted and configured with Windows XP SP3 and all the latest drivers from the Lenovo website, is freezing up on me around once every two days on average. 
At first I thought I had fixed this with the latest bios update but just got three more freezes since. I also thought it might be due to overheating as it froze twice on me while sitting on my blanket, but it also hung on me on my desk at work (underside was quite cool at the time). 
Not sure what the problem is. Googling the issue seems to suggest a lot of other owners having similar issues and no statements from Lenovo. I'm not keen on sending it in for warranty repair at this time as I'm sure they'll just send it back saying they could not reproduce the problem and three weeks later I'll be in the same boat.
I'll post an update here if I ever find a solution. Or if you're an X300 owner and have the same issue, please leave a comment below.
UPDATE: 2009-05-21 Installed Windo…

New Template with Customizations

I was previously unaware that you could upload custom templates to hosted Blogger accounts and was using one of the bland stock Google themes. I've since downloaded the Fresh Blogger Template, which is what you're seeing now (unless you're reading this in the future and I've upgraded to something even better).
The Fresh template is nice as it is (although a little too much blue) but I had to make a few customizations to get it working better: Converted the split two-column sidebar into a simple one-column bar.Made the searchbox a little wider to make the top-right-bottom margins symmetric.For some reason, Internet Explorer was trying to download my page when going to the home URL, added <b:include data='blog' name='all-head-content'/> just below the head tag to fix this.The RSS links in the template were inactive (#), changed the links to <a class='rss' expr:href='data:blog.homepageUrl + "feeds/posts/default?alt=rss"'>…

FreeMind Brain Export

I was previously using a local WordPress install as a Personal Knowledge Management tool to record everything of importance to my post-graduate research studies. The advantages of WordPress for this purpose is the non-hierarchical linkage of ideas and knowledge via tags, mimicking the way a human-brain works more closely than other flat logging structures (word documents, etc). This works quite well, especially due to the ease of inserting images, diagrams, links and formatting in WordPress posts. 
However, the linkage of ideas via tags is not well represented, you still need to remember or know what tags are meaningful, and how to get to a particular post you had. 
This is where FreeMind comes in, an open-source visual mind-mapper. Taking a look at the screenshots convinved me to download and try out this tool. Being presented with an empty map and a single node at first seems quite boring and useless. However, investing some time in learning how to use the tool, and especially the key…

Blogger Frustrations and Limitations

I've been using Blogger over the last three days to host this site. The choice between WordPress (hosted) and Blogger (hosted) was a tough one. I picked Blogger due to three main reasons:Free CNAME custom URL mapping;
Does not insert random ads in my posts that someone else benefits from;
Editing is somewhat faster than WordPress 2.7.1.
However, Blogger is really starting to frustrate me now, especially the primitive visual editor: When inserting images, they always get placed at the top of the page, regardless of where the cursor is when I click the insert image icon - I have to manually switch to HTML mode and copy-paste the ugly code chunk to the correct location.When inserting images, regadless of what size I pick, the image always gets inserted as it's original size - I have to manually edit the HTML code to change the thumbnail size.The image thumbnails are resized in the browser, meaning the full image gets downloaded regardless of whether the end-user has any interest in s…

Easiest Rubix Cube Solution

For those of you that think solving the Rubix Cube requires an IQ of 487+, you're probably still right. Solving the cube with no help is very challenging indeed and is a true test of one's logic (and patience). Luckily, a lot of people much smarter than you and I (or at least me, you might be one of them) have already done all the hard work and figured it all out. Most people who can solve a cube these days are merely learning and memorizing existing techniques and patterns, which is infinitely easier to do and doesn't take much skill at all (doing it blindfolded in under a minute is still something though).

There are a number of different methods typically used for solving the cube, the two most common being a layer-by-layer approach and the expanding corner approach. The expanding corner method is typically more difficult as it requires memorization of more patterns and more intuition as to what patterns are best applied when. The advantage however is that by memorizing m…

SuperSUM - Optus Broadband Usage Meter

SuperSUM (Super Simple Usage Meter) was a small program for Australian Optus Broadband Internet users allowing them to quickly and easily check their download statistics. The program lived in the system tray and periodically queried Optus with the user's account details retrieving usage statistics. Users would then simply hover the mouse over the tray icon to instantly see how much they've downloaded, or double-click the icon to see a bar chart of remaining peak/off-peak quotas.

Additionally, SuperSUM implemented a small, unobtrusive real-time download/upload speed meter which sat just above the system-tray showing exactly how fast the internet connection was. This feature supported multi-homed machines.

However, as of 15/01/2009 Optus killed the data feed driving this application rendering it useless. Other similar open-source applications were then recoded to screen-scrape the Optus web-sites to retrieve the usage statistics. I no longer have the source for this application, o…

KBSMan - .NET Keyboard Shortcut Manager

KBSMan (Keyboard Shortcut Manager) is a Windows only tool that allows you to easily assign and manage keyboard shortcuts for launching applications or executing commands/scripts. Additionally, the tool allows you to export/import your shortcut list to file for portability across work locations.

Key Features:
No installation required - just double-click the exe to run;
Starts on windows startup;
Easily import/export shortcut list;
Centralized shortcut management;
One-click activate/deactivate shortcuts (useful when you hit a conflict);
Add shortcuts to applications, files, paths, scripts or commands;
Minimizes to system tray;
Low CPU and memory footprint;

Usage: Download, extract to a directory and double-click KBSMan.exe run.

Requires: Microsoft .NET Framework 2.0+

License: Free for personal and commercial use.

Javinder - Java based file/text/class finder

Javinder (Java Finder) is a simple file search tool with some advanced capabilities. The tool can be broken down into three main functional groups: Class Finder, File Finder and Text Finder.

Class Finder: If you’ve ever struggled to resolve a ClassNotFoundException whilst trying to use 3rd party Java libraries, than this one’s for you. Javinder allows you to specify a fully qualified class name or a partial wildcard class name and a directory search path. The tool will then recursively scan all directories in the search path and look inside all jars for the given class, reporting all found matches. 

File Finder: As a natural extension of the class finder, Javinder allows you to specify a filename (with * and ? wildcard support) and a directory search path. The tool will then recursively scan all directories for all matching files, and will optionally scan inside all supported archives (zip, war, jar, ear). 

Text Finder: Similar to the Windows ’search-for-files-containing-text’ feature, o…

WordPress + bib2html plugin + JabRef

I'm using a locally hosted WordPress installation running on XAMPP as a Personal Knowledge Management system for my post-graduate research studies. I've also started using JabRef as my BiBTeX Reference Manager. JabRef stores its reference library data in a simple .bib file.
Since I'm using WordPress on my local filesystem, it means I can very easily configure JabRef to store the .bib file somewhere in my web-root...
The next logical step would be to use a WordPress plugin that can integrate with this .bib file so that I can have a full bibliography page, and additionally I'd like to be able to insert invidivual BibTeX references in specific posts.
Luckly, I think I've found a plugin that does just that. Bib2html simply parses the .bib file and adds some html formatting around the entries for pretty-printing in WordPress posts. I tried this with a simple .bib JabRef library and it didn't work however. I get the following error whenever trying to add [bibtex file=Ja…

Blogger needs a Label Cloud

One of my main gripes with Blogger is the simple line-by-line layout of labels in the gadget on the right. The gadget only has two options - sorting the labels alphabetically or by popularity. My problem with this is that labels are not categories. WordPress has a very clear distinction of this as it supports both categories and tags (labels):Categories are intended to organise your site content into a hierarchical tree with only one path from the root node (home page) to any child node or leaf. 
Labels or tags on the otherhand are intended as metadata that highlights the main semantic points of a particular page or post, thus giving you the ability to organise your content in a graph structure.A single page or post may have 5 or more tags or labels associated with it, but rarely just one. This means that list on the right is going to grow very quickly and thus a more condensed version would be more suitable.
WordPress and other Blogs/Wikis share a common component that shows all labels…

Lenovo X300 + Windows XP SP3

I recently purchased a Lenovo ThinkPad X300 to use as my primary computer as they had a massive clearance sale, managed to score one for half-price. I won't go into details on the machine's specs and features, there are enough reviews out there as it is. I'll instead just list some of my thoughts and experiences as I started using this machine.
Firstly, the machine came pre-loaded with Windows Vista Business. I've tried to stay away from Vista thus far due to all the negative feedback and because it doesn't provide enough motivation to switch from the stable XP Pro. Additionally, I see Vista as a poor choice for this machine for three reasons: The X300 runs on a 1.2GHz Intel ULV CPU, i.e. less processing grunt then typical laptops in the same price-range. Vista is commonly believed to be more processor intensive, even though this has probably been benchmarked to death and results sit on both sides of the fence. Nevertheless, I'd prefer to take my chances with a w…

DropBox + WordPress/Blogger

One of my major gripes with both WordPress and Blogger (hosted versions) is the restriction of file extension uploads. Blogger will only let you upload images to embed in your posts, whereas WordPress will allow images and a few different document types (doc, docx, pdf, ppt). WordPress also has an upgrade option in which you can purchase more space and with this upgrade you'll unlock the ability to upload common audio/video files (mp3, wav, mpg, avi, etc).

This is still highly restrictive however, what if I want to upload a zip file of an application I made and publish it on my blog? In the past you'd typically need to use a self-hosted WordPress installation for this which had no file extension restrictions.

However, DropBox has an interesting feature that easily allows you to create public links to any files you store in your public directory under your DropBox target folder, which you can then publish in your blog post. The procedure is as follows:
Write a post in your blog to…

WordPress + XAMPP + TrueCrypt + DropBox

If you've been following some of my earlier posts on PKM or PIM you'd be familiar with my attempts to use WordPress or Blogger as a knowledge management tool due to the ease of posting, chronological logging and advanced tagging support for easy information retrieval.
An earlier post discussed why these two blogging systems cannot adequately cater for mixed knowledge privacy levels, i.e. cannot have both public posts and private posts in the same system while retaining all the tagging, categorization and searching benefits. 
This forced me to use Blogger for my Public Knowledge Management aspects (i.e. this site) and use a different solution for my Private Knowledge, namely a self-hosted WordPress instance. I chose to self-host this as opposed to simply creating another freely hosted Blogger or WordPress account with the private global setting set to true for security reasons - my private information should only ever be visible by me (a restrictive model, I know).
Self-hosting my…

WordPress 2.7.1 Slow

Today I've tried using WordPress 2.7.1 both as a hosted service on and as a fresh install on a local WAMP stack (Server2Go). Both are running unexpectedly slow. Clicking on various links in the dashboard takes several seconds to load. Not sure what the problem could be as all other web pages are working fine, and all other localhost applications running on the same Apache instance are very snappy. 
I've been reading several posts in the blog-sphere with other users complaining of major slowdown issues with the latest WordPress. Various comments and suggestions are flying around but haven't come across any concrete solutions or explanations as of yet.
Note: enabling Google Gears in the Dashboard has no effect on performance, either in the hosted solution or on my local instance.

Public vs. Private Knowledge Management

If you've read my earlier posts from today you'd know that I've been experimenting with the idea of using a blog as a Personal Knowledge Management (PKM) tool. The main advantages of blogs for this purpose are the chronological recording of ideas mirroring a typical progress log that one might use to record daily ideas, thoughts, findings. Unlike a simple paper-based or Word-document like log however, blogs allow you to add labels or tags and categorize posts for easier information retrieval.
This lead me to have a look at both Blogger and WordPress as potential PKM solutions, however both came short when looking at private vs. public post permission handling. Basically, I'd like to be able to use the blog as a complete Knowledge Management tool, my one-stop-shop for recording and refering to anything and everything I know. Obviously, a lot of this information I would not want to share with the rest of the world (hence the Personal in PKM), but some bits may be generic …

Switching back from WordPress to Blogger

I love WordPress. It's much more customizable then Blogger, has better themes, gives you more flexibility, has static pages, categories and tags as opposed to just labels, and it just feels like a much more advanced system. 

Blogger in comparison seems quite primitive - the themes are bland, the customization options are limited, it only allows me to upload image files and nothing else, the visual editor is more basic (and buggy in Chrome it seems).
So why am I switching back to Blogger if I like WordPress better? Blogger offers a better hosted solution in my opition - free custom URL via CNAME mapping, no random ad insertion, seems slightly faster (probably because it's so much simpler)... 
The file-upload issue irks me, but I may use DropBox to host my public files and add links from this blog if needed. This should give me the control and flexibility I need going forward, I hope.