Good security pattern: Remote key

With Web 2.o, it is very likely that your browser UI functionality will also need to be exposed programmatically via an API with 3rd party software. Just like twitter can be used via tweetdeck, twhirl etc.

The user will still need to login to your site for the API call. In such cases, it is a good idea to have a separate password ("remote key") for 3rd-party tools, and a main password for the online login. What you should also do is to restrict the kind of activities that can be done after a remote-key login.

While you may be fully able to control the security of your site and your user's password, you cannot guarantee the behavior and security of 3rd party apps. With a separate remote key, the remote key can be easily changed if it has been compromised. In the interim, the compromised key can only be used for a restricted set of activities by the attacker.

We used this pattern in an Adobe AIR app we developed internally at work. Just noticed the same concept when I signed up for Friendfeed.

This looks like some of the applications at work

This is what design is about

"You know you've achieved perfection in design,
Not when you have nothing more to add,
But when you have nothing more to take away"
-- Antoine de Saint-Exupery

If you hire monkeys, you will get monkey-quality work

Found this in an email forward...

Same thing in software development as well. Programming is a combination of art and science. It's like the saying my grandfather once told me - "A 100 fools cannot outhink 1 wise man". A 100 monkey-coders cannot beat 1 great programmer.

Imitation is the best form of flattery

Or something like that...

It seems that Google is using CSS from the open-source Yahoo User Interface library in the Google personalized homepage service.

Word of the Day: Chernobyl Design Pattern

This gave me a good laugh. The Chernobyl design pattern as explained by Rob Weir (via cruizer) ...

"There is something I call the “Chernobyl Design Pattern”, where you take your worst bug, the ugliest part of your code, the part that is so bad, so radioactive that no one can touch it without getting killed, and you make it private and inaccessible, and put a new interface around it, essentially entomb it in concrete so that no one can get close to it. In other words, if you can't fix it, at least contain the damage."

The good thing in doing this is that it allows you to create a new implementation of the bug-ridden code later at your leisure. But we know that this never happens...

MapReduce plug-in for Eclipse. Now go and change the world...

Over the years, Google has developed some truly awesome "force-multiplier" architectural software components. These have helped them roll out hugely-scalable and high-performance applications very very fast at low low cost. Among them are -
-- MapReduce parallel computation framework
-- BigTable database
-- Google File System

MapReduce is a framework that -
-- Allows a programmer to define a program that needs to be run in parallel over a huge cluster of computers
-- Executes the software program in parallel by distributing it to a huge cluster, monitoring the execution progress and collecting the results

MapReduce has lowered the bar within Google for writing hugely-parallel applications. Mostly, the programmer has to worry only about writing the program in compliance with some MapReduce framework requirements. The entire parallelization and its enormous complexity is handled and hidden by the framework, freeing the programmer to concentrate on the actual functionality performed by his/her program. Within Google, MapReduce is used for building the search index etc etc.

Google has explained their MapReduce framework in this excellent paper by Jeffrey Dean and Sanjay Ghemawat. Based on this paper, Hadoop is an open-source implementation of MapReduce components.


Now, IBM has made available a plug-in for Eclipse that simplifies the development and deployment of MapReduce programs meant to be run within Hadoop. So all you brilliant engineers out there, go ahead and code the next parallel program that will change the world etc etc

Google is using Hibernate ORM? That's surprising

Update: Explanation of database partitioning techniques updated and enhanced. See towards the end of the post...

Update: Lots of people are coming here from the google-code blog. You might want to check out my posts on Simplicity, Management, Software Development, Career, or the main blog. If you like the content, you can subscribe to the feed. Thanks.


Learning that Google internally uses the Hibernate Object-Relational Mapping (ORM) framework brought my opinion of Google down by quite a few notches.


I strongly believe that in performance-critical applications, database transactions and SQL queries are best written and managed by the developer rather than by an over-generic ORM framework. If you have a low-usage application that will be simultaneously used by at the most 300 users, then go ahead, use an ORM layer and knock yourself out. But it is likely that if you have written a good app, it will become more and more popular, and you will start hitting performance problems soon. Any Google consumer application would be at the high end of performance-criticality requirements.

The reasons that have shaped my views are as follows -
-- Typically any application spends 25% of the time executing application code, and the other 75% executing database queries (Rough estimates based on my experience)
-- So if you want a high-performance app, optimize the data access layer
-- An ORM tool is very generic. It is meant to support multiple RDBMS systems and a huge variety of usage scenarios. So these are optimized for flexibility and not performance
-- Optimizations on the data access layer are best achieved by optimizing the SQL. You know your domain model and data schema best. So you are the best person to write the SQL
-- ORM tools like Hibernate do not allow you to write your own SQL. Instead you give definitions of your database tables and map them to your business objects. Hibernate automatically generates SQL from these relationships

Many people take issue with this and say that to manage transactions and write optimized SQL, you need very good developers, who are expensive and difficult to attract and retain.
My response: Of course, if you need to create outstanding apps, you need good developers. If you can't attract and retain them, that's your management problem. As for the cost aspect, hire a great developer, fire 5 average ones. You will get more work done, and save money.


Anyway, all the above is deviating from the main topic. Which is that Google has open-sourced Hibernate Shards, which is an extension to Hibernate to enable Horizontal Partitioning of Data.

What is Horizontal Data Partitioning? Well, if your app requires access to large large amounts of data (hundreds of millions of records), and performance is critical, and you have a large number of users, then storing all your data in one database will hit problems. This is because there will be too much read/write to the same database storage. One solution is to split the data in a big database table into "horizontal partitions" based on some particular criteria, and store the partitions in separate databases. For example, you will group your user profile table based on the user's location - US, Europe, Asia - and store each group of profile data in a separate database. This is called horizontal data partitioning as you are splitting the data at the "row level" within a database table.

Another option is Vertical Data Partitioning. Here, you partition the data in a database by putting different high-volume database tables into separate databases. For example, if you have lots of User Profiles, Messages, Transactions etc etc in separate tables, then with vertical partitioning, you will store each of these tables in separate databases. This is called vertical data partitioning as you are splitting your data at the "table level" within a database.

"Email This" Hall of Shame - NewsGator Online

For context, see my earlier post - "Email This" - The cheapest and simplest way to attract new users


Now on to today's star...


Yes. Ladies and Gentlemen. Removing spaces b/w the 2 addresses thru JavaScript seems to be an NP-hard problem.

"Email This" - The cheapest and simplest way to attract new users

In order to grow your web business, you need new users.

There are a multitude of ways in which you can attract new users. Targeted ads, affiliate programs etc etc etc. But the cheapest and simplest way is to let your current users do it for you for free.

For this to happen, you need to do 2 things -
-- Create compelling content. It should be compelling enough that a portion of your users want to tell others about it. This is the critical path. Without this, there is no point trying anything else as otherwise any new users you manage to attract will ultimately leave.
-- Once your users find something on your site compelling enough to share, make it easy for them to tell their friends about it

I started using the Internet in 1999. In trolling thru countless news-sites, blogs, web apps etc etc since then, the easiest way I have found to share something is thru an "Email This" link right next to the content I wanted to share.

It in incredible how many people manage to botch this up. So, from my user experience, this is my cheat sheet for a great "Email This" link design...
-- Have the "Email This" link displayed prominently next to the content you want your users to share. Having a familiar letter icon will help. The aim is to put the thought of emailing the content into the mind of a user who has not even considered sharing your content.
-- Clicking the link should preferably not open a new page. The best implementation would open a new Ajaxy-layer within the same page. The aim is to prevent the user losing interest in the time it takes to load an email form in a new page.
-- Email the entire content. Most recipients don't bother clicking on URL's sent by their friends. If the content they read in their mail client is good, they will come to your site looking for more.
-- If you gotta have a captcha, then try not to make it intrusive. I hate captchas that are filled with numbers because I need to refer to the captcha again and again as I fill it out. As I have mentioned in a previous article (Corvette not GS450H), human beings prefer meaningful words rather than numbers. If it takes too long to type the captcha, I'm outta there.
-- Don't have stupid restrictions in your email form, like allowing only 1 recipient address, enforcing ridiculous formatting requirements between multiple email addresses etc etc.

In the next few posts, I will highlight some of the most brain-dead implementations of the "Email This" link that has had me banging my desk in frustration. I will also cover my search for the perfect "Email This" link for this blog.


Related Posts
Corvette not GS450H
Why "Simplicity Works Everytime"

Samuel Jackson's Pulp Fiction shooting scene in true type

Watch this with the sound cranked up...
Awesome! Pulp Fiction is one of my favs.

When I think about it, if I were running an interface-design or interaction-design firm, these are the people I would want to hire. Font type designers for whom type design is not just a job; but a calling.

Windows Media Player 11 sucks

Is it only me who finds WMP 11 among the most atrocious software out there?

1.) I can't find the friggn menu bar
2.) The whole thing takes ages to start up and freezes whatever is running on my PC at the same time
3.) It sucks up so much memory
4.) There is some fundoo menu system running from left to right that is very very unintuitive

My suggestion to Microsoft. Instead of getting program managers to come up with specs to catch up with iTunes, please do this. Allow 1 talented developer in your Media division to take 2 weeks off. Let him sit at home, code and come up with his own version of WMP 12. Its gonna be loads better than what your (Media GM + Media Product Unit Manager + WMP Group Program Manager + army of WMP Program Managers) can come up with. Since MSN Messenger has around 40 program managers, I am guessing there are loads more for WMP.

Great software does not come from Program Managers, Business Analysts or Interface Designers. Great software is visualised by craftsmen. And the craftsmen in software are the developers.

Why "Simplicity Works Everytime"

Some explanations on the title of this blog...

From my work in delivering, designing, and developing high-performance financial systems, I have been struck by the fact that the most usable, most loved (yes that is possible), best performing, and highly scalable systems have simplicity as their underlying theme.

Now simplicity can be a highly subjective topic. So let me clarify what I mean...


You can broadly classify any system into one of these two categories -
1.) A User-Driven System. This is any system with a customer-facing user interface. (Technical support user-interfaces do no count here)
2.) A Technical System, i.e. a non-User-Driven system. This could be a server process that runs independently of any customer interaction.


In the presence of competition, the success of a user-driven software application is a function of how much the customer likes/loves using it. This is exponentially proportional to it's the ease of use. For most users, software is just a tool to get some work done. It is not a primary part of their existence. Users love software that allow them to finish their task with the least amount of thinking so that they can focus on the really important things in life. So for user-driven software, simplicity equates to "simplicity for the user", not "simplicity for the developer". In order to provide this simplicity to the user, the underlying technical design might have to be complex, very complex. Sorry, there is no way around this.

And finally, user-driven software attain the nirvana of simplicity when they become humane. These software forgive our human follies, and guide us ever so subtly in our tasks.

Let me tell you why I absolutely love iTunes. It has this great feature called "alphanumeric sorting". In layman's terms, this means that a song titled "Eagles - Track 10" will come after "Eagles - Track 9", and not "Eagles - Track 1" as happens in Windows Media Player. iTunes also has another amazing feature where they discard the word "The" in the band name during sorting. So "The Cardigans" comes under C, not T.


For pure technical systems, things are different. The best technical systems have very very simple technical designs. Great technical systems do not have highly normalised data schemas; Simple technical systems are not in love with their reflection; Simple technical systems actually hardcode things that will never ever change; Simple technical systems do not try to retrieve data using humongous object-relational mapping layers; Great successful systems mostly never use EJB's; Great systems solve one thing very well.

In DB, we bought a 3rd-party scheduling software that used 90 Oracle tables internally for managing clustering. The thing bombed in actual production usage. An internal team rewrote the same functionality with some simple code that used 9 Oracle tables for managing a load-balanced cluster. It is so successful that other teams in DB are using it, and we are thinking of making it open-source.


The problem is that as an industry, we have completely lost the concept of simplicity. We have become experts in abstracting any user request into a generic problem, and spend ages building generic systems that look great to us in our mental sheet of paper, but hideously complex at runtime. We have forgotten that software is actually about working applications. Instead of solving user problems, we have all become self-anointed architects.


One of the sidey things I will be doing thru this blog is to keep up a steady stream of views and observations on keeping things simple.


"You know you've achieved perfection in design,
Not when you have nothing more to add,
But when you have nothing more to take away"
~ Antoine de Saint-Exupery