Centralization


Wednesday, November 28th, 2007 at 6:10 pm, philosophy

Last weekend I traveled to a place where it looks better than sunny Switzerland normally does. Though thats not very hard as you can see

Swiss weather

Wonderful things those trips to Las Palmas by the way. I strongly suggest to anybody who has the time and lives somehow close to go.

Anyway, I’m not talking about the sunny days I spent there. The really great thing was that I met an old friend I thought I had lost years ago. He turned up out of nothing. So suddenly that it took me few moments until I realized that it was really him. But there he was again, origin of pleasure and amusement, fountain of long and intense nightly discussions. Let me introduce him here!

TV

He even made it to TV. Boah!

Mixing Data

The problem we see here is not just the result of a crazy guy who put his long running service on an OS which is notorious for getting slow and unstable after using it for a long time.

I think it gives an adequate example of how putting everything into a central place may mess up everything. If you save all you have into those database oriented file/configuration pools you give away all the control you have about your data and thus your application as well.
Of course while saving your application’s data you may receive an error and the next time you have an invalid file as well, but at least you can somehow handle that (by holding tmp files or whatever) If your data storage gets corrupted you may not even be able to access the incomplete data at all.
Moreover, you are not the only one who is accessing that storage. When the power cord gets plugged out and hundreds of other clients are saving their stuff to the same place its way more likely that the internal structure of that storage gets corrupted (more likely, it may be avoided of course).
Then you face the problem that it was maybe your data which is not accessible anymore after the next reboot because some other guy was saving his stuff, accidentally spoiled the data from all other clients and you can do nothing about it.

In the Windows world there are a few failed DB-design based features:

Vista Filesystem (WinFS)

The guys at Redmond wanted to store all files in a huge database-like filesystem. You have transactions, can scale over several computers, everything! The real Wow effect!
Almost finished However, it did not ship with Vista and I don’t know what happened to it. I guess it’s imploded under it’s own weight. Maybe it was too slow, to buggy or whatever.
You can ship any application with some bugs but a filesystem has to be perfect. One small bug and data starts to disappear in random places and will make your users running away in droves.

Anyway, it seems it’s not trivial to correctly implement a storage with a DB design so it still achieves good performance and provides the essential features of a filesystem, otherwise it would have been shipped.

Registry

Another nice example of what happens when data from different clients get mixed together.
It is the last resort of uninstalled applications and everlasting producer of problems. One wrong edit and your system doesn’t boot anymore. Moreover, unofficial waste bin of unsorted configuration data for all kind of applications. Not to compare with the relatively clean structuring of a multi file /etc directory.

Outlook/Exchange

Normal users tend to use their mail account as a searchable file storage. Nothing wrong with that. They just noted that the normal windows search is crap and in their mail program they can tag their files with emails and separate them in categories. The only guy who is not so happy about that and responds with mailbox size limits is the one who has to maintain all those things. He has to do that because all mails get stored in a gigantic file and when that file exceeds the 16GB storage limit he has a problem. (If he wants more he needs an expensive update, has to hack the registry etc.) Why not providing a unix like mail spool with a single file for every user? Another example of how you create problem by storing everything in the same place.

Everything Is A File

The unixen philosophy where everything from a socket, to handles, to drivers is a file is the much simpler and thus more powerful and flexible way than inventing a new architectural paradigm for every emerging problem you face. Furthermore, coming up with a new approaches for such basic things instead of using a proved abstraction model just introduces unnecessary complexity which will sooner or later (maintenance) cause tremendous headache.

Trouble

By putting your code into dependency of others you increase the probability that it fails. This is especially true if your dependency is accessed by other clients as it is done in those file/configuration databases. Every external component you use may fail and typically you have no way to fix it. There are already enough problems an application has to face. By requiring yet another fancy module you just add another hurdle which could break your program.

It’s that simple.

6 Responses so far

  1. giu Says:

    I would say, that the registry is the same sh** as windows millenium..oh wait! in windows millenium you’ll find the registry, so windows millenium is the bigger sh** :)

  2. Chrigi Says:

    I agree with you here steve, single file based tend to cause less headache… Just think about the xml configuration hell we introduced in our app. Clear boundaries where what is stored would have simplified many things.
    Though when it gets too fragmented you have even more problems as in the first place… so like usually in our job, you need a feeling what to do.. won’t learn that by reading books.

    btw, you see reto, we pay him too much. That euro-jet-setter, was on holidays again:

    we flying first class,
    up in the sky,
    popping champagne,
    living the life

    G-L-A-M-O-….

  3. giu Says:

    we flying first class,
    up in the sky,
    popping champagne,
    living the life

    G-L-A-M-O-….

    stop this! he can’t stop singing it @ school, our glam-boy :)

  4. Reto Bachmann Says:

    …who put his long running service…

    aka “PayTV”, 00:15 in the morning. Steve, Steve.. what were you doing then ;)

    Anyway, I fully agree with the other things you wrote. Keeping control is a must or your product slowly becomes an unmaintainable monster.
    But about that TV: It could have been also be a temporary down for a few minutes.

  5. luca Says:

    aka “PayTV”, 00:15 in the morning. Steve, Steve.. what were you doing then ;)

    Heh, you see how dangerous it is to publish so many details.. eh :)

    but nice post. thx! though you should also mention that you dislike dbs in general and thus its clear from the beginning what you are going to say :)

    hope to read something new soon.

    ciao

  6. steve Says:

    @giu
    WindowsME. Those were the days! I have to admit though it didn’t seem too terrible for me back in the days when I switched from win95. (dark days then, I didn’t even know that there was linux)

    @reto

    aka “PayTV”, 00:15 in the morning. Steve, Steve.. what were you doing then

    nothing :)

    It could have been also be a temporary down for a few minutes.

    Nope. I was in Las Palmas 2 weeks before that and they had their TV-problem also then.

    @chrigi
    You got the link to the real world :p I would say our solution is a good balance between the aspects you listed.

    @luca
    pssst! thats still a secret between us.

    @fergies
    It will always be about the glamorous, the glamorous glamorous. About the glamorous… ouhh!

Leave a Reply