Tools

2011-10-30T00:00:00-07:00

As long as I can think, Computer Science has been infected by religious flamewars. Having arguments about technical topics can be healthy but flamewars are not. I’m sick of it. I’m fed up by people telling me that their work environment is oh-so better, faster and so on. That’s fine but it doesn’t matter. Your equipment only plays a supporting role. You don’t even need a computer to do programming. Donald Knuth wrote algorithms on a notepad. Alan Turing wrote the first chess computer on a piece of paper. And it worked. Beat that!

For an average user, the next best system is probably good enough. Just a few bucks and you get a fine piece of hardware which is completely sufficient to surf the web, chat, archive photos, write documents, listen to music and watch movies. You can do that with a Pentium IV, 256 MB RAM and any recent Operating System (you will likely get that one for free). Heck, you can use your old Commodore for most of that. Computers have been mature and reliable enough to do all that for ages. There’s really no need to upgrade your system for Farmville, just like there’s no reason to buy a new car if the old one works perfectly fine. When it comes to software, many of us still use Office 2000 or Photoshop 8 or VisiCalc without feeling the urge to upgrade.

Professionals find themselves in a similar situation. Well, maybe we invest a bit more money but still our hardware is incredibly cheap compared to our salary (hopefully). Nothing is perfect, but most of the time it’s good enough. That compiler you were using a decade ago? Still does the job. We are still using slightly modified descendents of programming languages from computing stone-age. Even if you’re doing numerical computing for NASA, your basic work environment is a black box running a text editor or an IDE.

I don’t care what you are using to get things done. Find an environment that suits your needs and be happy with it. Maybe you use Emacs on a Lemote Yeelong netbook (hello Richard Stallman) or Vim on your workstation. It’s basically the same thing: A text editor running on a piece of metal. You’re not a worse programmer for using Nano, ed or TextMate. Notepad works just fine, too. It loads files, saves files and lets you edit them in between. That’s a hell lot more functionality than Bill Gates and Paul Allen had when they wrote a BASIC interpreter for the Altair. If you find something you’re happy with, just stick with it but don’t start arguing. It really isn’t worth your time.

Don’t feed the trolls. When it comes to software, don’t fall into the old FreeBSD vs. Linux vs. Windows vs. mum cliche. Instead talk about your code. Let’s look at your problem solving skills. Let’s be pragmatic here.

“Talk is cheap. Show me the code.” - Linus Torvalds

I don’t care which programming language you are using. Java? Fine. Visual Basic? Great! Scala, Cobol, PHP, C++? All fine. Write in Assembler or lolcode. Don’t mourn about the fact that language X is missing feature Y. Write a library or use something different. Stop saying JavaScript is a toy language. It just doesn’t fit your needs. Instead show me your Lisp adventure game. Write an interpreter for Brainfuck. Do something. Move things.

Concerning PHP, nir wrote on Hackernews:

Any idiot can write a snarky comment about PHP. Very few get to write code that has anywhere near the impact it had.

Will you fall off your chair when I admit that I like the PHP syntax? OK, it has its rough edges (do we really need the $ sign?) but what’s more important is how much I can get done with it. PHP was my long time go-to language for off the hook, one time scripts. It looks a bit ugly but it runs on any server and comes with an enormous amount of built-in functionality. It’s great for rapid prototyping and gluing things together. In fact, when you write a piece of software, what you should strive for is to produce quite good software and what you really need to accomplish is good enough software to make your users happy.

Zed A. Shaw puts it quite nicely in the afterword to Learn Python the hard way:

I have been programming for a very long time. So long that it is incredibly boring to me. At the time that I wrote this book I knew about 20 programming languages and could learn new ones in about a day to a week depending on how weird they were. Eventually though this just became boring and couldn’t hold my interest. What I discovered after this journey of learning was that the languages didn’t matter, it was what you did with them. Actually, I always knew that, but I’d get distracted by the languages and forget it periodically. Now I never forget it, and neither should you.The programming language you learn and use does not matter. Do not get sucked into the religion surrounding programming languages as that will only blind you to their true purpose of being your tool for doing interesting things.

Don’t get emotional for any tool you use. An iPhone - I’m sorry to disappoint you - is just a phone. No magic. No “think different”. “But it’s evil!”, the ether says, “it’s not open source”. Well, Android just exists because Google needed to rapidly develop a mobile platform. It’s simply part of their business. There is no moral behind that. Google is a yet another company just like Microsoft or Apple.

My MacBook serves me as a good tool but if something “better” comes around I will happily kick it out. I’ve ditched Firefox after five years just because Chrome is faster and I will get rid of Chrome when I find a worthy successor. Vim is quite good in my opinion but if there’s a faster way to do things I’m not afraid to dump it. Instead get your hands dirty and fix the problems or craft something new.

Are You A Programmer?

2011-10-20T00:00:00-07:00

My geography teacher once told the story of her first university lecture. As an introduction, her professor asked the class to draw a map of Germany without any help and as accurate as possible. To her surprise she was not able to fill the map with much detail. Even the shape of the country was a bit vague.

She had seen thousands of images of Germany (her mother country) but wasn’t able to reproduce it from her blurry memory. She would have to look it up.

Doesn’t this sound familiar? We rely on machines to manage large potions of our knowledge. There’s hard work involved to learn something by heart.

Here is a similar test for programmers:

Using a programming language of your choice, write a correct sorting algorithm with an average runtime complexity of O(n*log n) (Heapsort, Quicksort, Bucketsort, you name it) on a piece of paper without the help of any external tools.

And by correct I mean it must be free of bugs without any modifications when you type it in.

You would be surprised by the large percentage of professional software engineers who can’t pull this off.

Some might argue that knowledge about details of programming language syntax is unimportant: “Why learn all the little nitpicks when you know how to use a search engine? Why start from a blank sheet when you can easily copy, paste and modify an example from a tutorial? Every few years/months I have completely relearn the syntax for a different language anyway.”

But that is a myth. If you know only one programming language really well - even if it is something outdated like Fortran or COBOL - you could easily earn a fortune with that knowledge. Suppose you started with C in 1975. You could still use the same syntax today - almost four decades later. Same for text editors. Emacs and Vim are both decades old. They are battle-hardened. I don’t care which one you prefer but you will spend most of your life with these tools so invest the time to use them really well.¹

If you don’t know how to use your tools without a manual, you are a bad craftsman. If you need a dictionary to write a simple letter, you will have a hard time being a writer because it would already be challenging for you to form elegant, fluent sentences – let alone engaging and original stories. I don’t want to read these books.

What makes a programmer?

She has at least one programming language she knows inside out.
She is able to implement standard algorithms (i.e. for sorting, searching) and data-structures (i.e. trees, linked lists) which are robust and reasonably fast on the fly.
She has at least a basic understanding of complexity theory and programming concepts like recursion and pointers.

But, to be a good programmer, you should

Be able to code in at least two fundamentally different programming paradigms (i.e. declarative, functional).
Have experience with big software architectures.
Be familiar with your programming environment like the operating system and a sophisticated text editor of your choice. Preferably one, that is easily extendable.

And that is just the tip of the iceberg. “There’s too much to learn!”, I hear some of you say. Start slowly. You need only three commands to start with Vim: i, ESC, :wq. That’s enough for day one.

I realize that most of these essentials won’t be taught during lectures. You have to learn a huge portion on your own. But let’s face it: If you don’t know this stuff, you are not a programmer, you’re a freshman.

It seems that very few people strive for perfection in anything they do. They happily settle for “good enough”. This can have many different reasons and I’m not blaming anybody for not doing his homework but maybe I’m not alone with that observation.
↩

On hard work

2011-10-13T00:00:00-07:00

Great people get shaped by their achievements:

There’s Thomas Edison who developed countless protoypes before selling a single light bulb.
The unemployed Joanne K. Rowling writing Harry Potter in a Café while caring for her child.
Steve Wozniak creating the first personal computer in his spare time while working at HP.

What do they have in common?

They all lived through frustration and contempt but still reached their goals, even though the chances for success were low. These people are stemming their strong will from an intrinsic curiosity.

Dedication

Sure, I love what I do. I want to be a programmer for the rest of my life, but sometimes it seems simply too hard to finish a project. I get scared by the big picture and fear that I won’t finish on time. What I need is a different mindset.

Dhanji R. Prasanna, a former Google Wave team member made this observation:

And this is the essential broader point–as a programmer you must have a series of wins, every single day. It is the Deus Ex Machina of hacker success. It is what makes you eager for the next feature, and the next after that.

While Google Wave has not been commercially successful, it sure was a technical breakthrough – and it was a drag to push it out into public. We always have to see our goal right in front of us, as we take a billion baby steps to reach it. This is true for any profession. Winners never give up.

Direction

Today it is easier to accomplish something meaningful than ever before.

If you are reading this, you have access to a powerful instrument – a computer with an Internet connection. We live in a time where a single person can accomplish miracles without hard physical labor. A time where billions of people can grow a business from their desk, get famous in minutes, publish books in seconds and have instant access to large amounts of data. The most powerful development over the last 100 years has been the reduction of communication costs. Transferring a bit of information to the other end of the world is virtually free and takes fractions of a second. While proper education was a privilege of a lucky few well into the 20th century, learning new things is now mostly a question of will.

Photo by Zitona (CC BY 2.0)

Nevertheless, learning is still a tedious task, requiring patience and determination ¹. As the amount of information has increased, so have the kinds of distraction. Losing focus is just a click away.

Devotion

Everybody can start something. Few will finish anything. That’s because getting things done is hard, even if you love what you’re doing ². No matter what they tell you, achieving anything sustainable means hustling. It means making sacrifices. It means pushing through. It means selling something even though it isn’t perfect. Your elegant project might turn into an ugly groundhog in the end. Put makeup on it and get it out the door.

On a report about Quake’s 3D-Engine, developer Michael Abrash says:

By the end of a project, the design is carved in stone, and most of the work involves fixing bugs, or trying to figure out how to shoehorn in yet another feature that was never planned for in the original design. All that is a lot less fun than starting a project, and often very hard work–but it has to be done before the project can ship. As a former manager of mine liked to say, “After you finish the first 90% of a project, you have to finish the other 90%.” It’s that second 90% that’s the key to success. […] A lot of programmers get to that second 90%, get tired and bored and frustrated, and change jobs, or lose focus, or find excuses to procrastinate. There are a million ways not to finish a project, but there’s only one way to finish: Put your head down and grind it out until it’s done. Do that, and I promise you the programming world will be yours.

That last part has influenced me a lot. The dedication, the urgency to reach your aims must come from within you. It’s your raw inner voice speaking – don’t let it fade away. And when you are close to giving up, stop thinking so hard. Just try to push forward and make a tiny step in the right direction. You can’t lose.

If you allow me to dream for a second, I presume that during the next few decades, this last hurdle might also be eliminated. Learning would simply be a matter of uploading files onto an organic storage device. Naturally, automatic learning will be a privilege of rich and adventurous people in the beginning. But just as space tourism, I am convinced that it will be accessible to everybody in the long run. On the other hand, mere knowledge is useless. It is applying and adapting knowledge to new situations (creativity), which sets us apart from machines.
↩
Watch the beginnings of There Will Be Blood and Primer for a definition of hard work.
↩

An open letter to Apple

2011-10-01T00:00:00-07:00

Since I work on a computer for most of the day I like to have the darn best thing I can afford to get my stuff done. For the last five years I have exclusively used a MacBook and afterwards a MacBook Pro and I have no doubt I paid a premium price for a premium product. Yeah, I’m also paying a premium price for every RAM extension, Mini Display Port - DVI adapter and keyboard made by Apple, too…and I’m totally fine with it.

Most of the time MacBooks are great machines, fun to work with and well worth the higher price tag. Some parts of it just stink, though.

Today I had a nice conversation with an Apple advisor (as they call them) because my power supply is broken. Again.

So here is a picture of the power supply:

You can clearly see where the cable is broken. It’s right at the spot where the protective sheath is ending.

I know this is the place where the cable is bent when it gets rolled up. The advisor basically told me that I have to pay the price for being too stupid to roll up a cable. But I think that’s the way the cable is supposed to be rolled up according to all their advertisement images.

Let me clarify: My current machine is a MacBook Pro bought in April 2010, so I’m out of warranty. Apple is not obliged to compensate. A replacement would be a gesture of good will. On the other hand I consider this a manufacturing error and a security risk.

Oh and while we’re at it, the new L-shaped magsafe plug is also badly designed. The whole purpose of that thing is to prevent damages when somebody stumbles over the cord. But unfortunately the protection mechanism won’t work if the person is coming from the wrong side. Here’s a picture that explains what happens:

Come on Apple. Hardware defects and design issues are normal. This shit can happen. But when it’s clearly your fault don’t blame your clients. I’ve invested a lot of money into your products during the last few years (both for private and corporate use) and I don’t hope you want to loose me as a customer.

Seems like I need to buy a new one. The fourth power supply for only two laptops. But nevertheless I’m sending my power supply back to you. Do with it what you want. Rant over.

My experiences with Feedback Roulette

2011-08-12T00:00:00-07:00

Feedback Roulette is awesome. The page reviews are honest and detailed. Here are a few comments I received so far:

The site is slow, not sure if it’s my connection, but response time is high.

This tip helped me dig a little deeper into SEO (despite I passionately hate the term). I’ve minified the CSS and Javascript files, used spriting and am now using a CDN, namely Cloudflare.

Was expecting the penguin to click and lead me to the home page.

This was a quick and easy fix that can be done in a couple of minutes. I didn’t realize that and was happy when somebody pointed it out.

On your movies page you could link to Amazon with an affiliate code, that way if people buy the movie on amazon you get some money.

I see where this is coming from but on my private homepage I’m not out for the quick buck. I was thinking about creating a wiki for the genre, though.

“Projects” areas are always tough when you when you only have a few active projects. May be add a github activity feed.

Since then I’ve added some more projects but the activity stream is definitely something to think about.

The site doesn’t load quite right in Opera. It’s hard to explain, but above the link for “Blog” it says “Blog” again in another box. This box stays across all the other pages (but it’s always “Blog”).

This behaviour is quite weird. Until now I couldn’t find out what causes it. It may well be an Opera bug but since I’m not an Opera user and this is a tiny private homepage I didn’t bother too much.

The links appear on the right-hand side and then the search box appears, pushing them to the left.

The “pushing” is caused by an ajax call to Google search: As soon as the search script got loaded, the search box gets added to the DOM. Naturally the solution was pretty simple. I swapped the search box and the links.

I think the text in the articles should be a bit smaller, I just find it slightly too large for a lot of reading.

I like big text. Nevertheless I had to admit that it was just a little too big.

All these comments came from real users and it would have been hard to find such responses by asking my friends. So the bottom line is this: If you have a little webdesign project and want some quick user reviews without all the hassle of A/B testing, Feedback Roulette may be what you are looking for.

Digg – a case study for the social web

2010-10-27T00:00:00-07:00

We are only at the beginning of a personal web. Presently social communities are humble experiments with lots of room for improvement. Twitter, Facebook, MySpace they all made failures along the way. They test different approaches to filter, personalize and rate the endless stream of information and it’s just natural that they don’t get it right the first time.

The tricky part is to gain new users while making sure that longtime members won’t opt-out. A few minor changes can lead to a hazardous outcry and a persistent loss of faith in the platform. Just recently the social-bookmarking site Digg suffered from these powerful group dynamics.

Taking the wrong way

In the beginning the site had a strong focus on tech news. It played an important role for the geek and hacker culture. People posted the AACS encryption key on that page and shared photos from Paris Hilton’s hacked phone. But in order to grow, they opened up to a broader audience featuring news about sports and politics. Diggs’ reputation as an authority on tech news suffered from that move¹.

Experiencing high pressure from a close-knit community they strived to innovate, creating a huge plan to reinvent web-publishing which turned out to be a misguided strategy. They’ve made the rating system too complex and opaque, removed important features (like the upcoming section and the bury-button) ignoring their loyal user base that favored grassroots democracy and disliked the extensive modifications.

That said, Diggs intentions were honorable. In order to deal with the ever-increasing demand, they’ve created a sophisticated distributed system powered by a non-relational database (Cassandra) and a personal front-end called My news. When Kevin Rose explained the new system, I was quite enthusiastic. But the (database) transition was not flawless and the focus on mainstream outlets was in my opinion a bit misguided. Some former power users even speculated whether Digg became a promotion platform for newspapers. While these accusations may be wrong, it shows the core problem: Proficient users don’t trust Digg anymore.

On the Digg comment thread to the above story, endersgame put it like this:

Since the launch of v4, Digg has lost all credibility and believe it or not that is very important. The reason Reddit is so successful is because the users feel like they are a part of the site. They feel like they have direct control over how the site works.

Take away the upper twenty percent of contributors to any social web-app and all that remains is a wasteland of shallow stories and comments — web mediocracy. On an aggregate news site like Digg, this could mean that the stories are already a few days old when they hit the front page. Therefore lively discussions – the main product of crowd sourcing – take place on other websites. Fewer comments lead to fewer visits which lead to even fewer comments. A vicious circle that eventually destroys a great community. So maybe Digg took the wrong way but I really hope they can recover. A lighter, more transparent version might be the answer.

Up next

After seeing all this happen to Digg I’m curious if Reddit and Hacker news can cope with the upcoming traffic explosion². While the former is a respectable bookmarking platform, I personally prefer the latter for the profound discussons on technical topics. Basically it’s a single table with links to great ideas and inspiring projects. In contrast to many other social bookmarking sites, there is a lot less comment noise (although the babel/visitors ratio may be roughly as high as on /r/programming). The hive mind recognizes most trolling attempts. The filter systems are not perfect but quite good.

Lessons learned

All websites are constantly exposed to the whims of their users but tech sites – with a lot of their regular visitors being geeks and hackers – have a particularly difficult position. Hacking is all about learning from others and making contributions to the community. It is about game mechanics, curiosity and reputation – not money. A hacker notices when his skills are abused – and moves on. These are the reasons why hackers are happy to answer questions on StackOverflow but shy away from ExpertsExchange.

Create a fair, simple, open system and hackers will bring the rest. That’s what Paul Graham did with Hacker News. He looked at some existing social bookmarking sites like Reddit, ripped out all the superfluous parts and iterated from there. Revenue was never on the todo list. No shiny design, no pictures, no videos, no media outlets. It has the flair of an early Slashdot.

It’s an Anti-Digg if you want. It makes lots of minor changes instead of abruptly releasing a whole new system like Digg did with v4. It’s pure KISS. A minimalist approach to social bookmarking. It does not try to be popular. In fact most regular visitors would be happy if it stays small³.

But rest assured there are roadblocks ahead. While the community will grow, quality of content is endangered to go south. Let’s see how long this will work out⁴.

Facebook was in a similar situation when they opened for non-students. But in contrast to Digg the outcome was not fatal (actually quite fertile).
↩
Subreddits are quite nice to canalize the information overflow. I propose such a system for HN looking at the increasing number of news that make the front page.
↩
It was initially a side project to test an experimental programming language called Arc.
↩
Presently I’m quite happy even though I’m spending more and more time on alternative interfaces.
↩

Importing old Wordpress backups into Intense Debate

2010-10-14T00:00:00-07:00

My new blog is pretty much up and running now. The only thing I was missing were my old comments I had lying around in a MySQL dump from my old Wordpress blog I made quite a few years ago.

An entry of the dump looks like this (in fact there’s one database entry per line which makes parsing easier):

INSERT INTO `wp_comments` VALUES (6, 41, 'mre', 
'matthias-endler@gmx.net', 'http://www.matthias-endler.de', 
'89.51.251.189', '2006-09-20 17:09:38', '2006-09-20 15:09:38', 
'Thx. Took me about three hours to get everything together. 
Good to see that it was worthwhile.', 0, '1', 'Mozilla/5.0 
(X11; U; Linux i686; de; rv:1.8.0.7) Gecko/20060909 Firefox/1.5.0.7', 
'', 0, 1) ;

(I can’t believe how long ago it is. I was using Firefox 1.5 at that time…at least it was faster back than ^^)

At present Intense Debate only offers a comment import feature for existing Wordpress blogs. But luckily Jay Graves over at Socialist Software offers a nice Python script that can send the comments to their system. I’ve created a Python 3.1 version of it:

import urllib, urllib.request, urllib.error, urllib.parse

def postComment(blogpostid="", acctid="", anonName="", anonEmail="", 
                anonURL="", comment=""):
    # "127889" # id test page blogpostid
    # "1762" # id test page acctid

    server = "http://www.intensedebate.com/js/commentAction/"
    data = '{"request_type":"0", "params":{    \
        "firstCall":true,                      \
        "src":0,                               \
        "blogpostid":%s,                       \
        "acctid":%s,                           \
        "parentid":0,                          \
        "depth":0,                             \
        "type":0,                              \
        "token":"",                            \
        "anonName":"%s",                       \
        "anonEmail":"%s",                      \
        "anonURL":"%s",                        \
        "userid":undefined,                    \
        "token":"undefined",                   \
        "mblid":"",                            \
        "tweetThis":"F",                       \
        "comment":"%s"}}'                      \
        % (blogpostid, acctid, anonName, anonEmail, anonURL, comment)
    params = urllib.parse.urlencode({'data': data})
    try:
        f = urllib.request.urlopen("%s?%s" % (server, params))
    except IOError as error:
        print("%s\n%s" % (error.code, error.read()))

Finally I wrote a parser for the SQL dump of my old blog and used the code above to send the extracted comments to Intense Debate:

import intensedebate

def main():
    file = open( "dump.sql", encoding="utf-8") 
    
    id_old  = 123       # Post id from old Wordpress blog
    id_new  = 12345678  # Intense Debate post id found in firebug DOM
    id_acct = 123456    # Intense Debate account id. Get it with firebug
    
    for comment in comments(file, id_old):
    	intensedebate.postComment(blogpostid=id_new, acctid=id_acct,  \
            anonName=comment["author"], anonEmail=comment["email"],   \
            anonURL=comment["url"], comment=comment["content"])
       	print("Posted ", comment)
        
def comments(file, id):
    ''' 
    Returns next (non-spam) comment 
    from a wordpress database sql dump 
    '''
    
    # Field names of the wp_comments table inside the SQL dump file
    marker, post_id, author, email, url, ip, \
    date, date_gmt, content, karma, spam = range(11)
   
    
    for line in file:
        # Split the line into its field values
        field = line.split(",", 8)
        
        # The comment needs some extra treatment
        # because it can contain commas inside the text:
        # Extract comment from last field in line dump
        last_field = field.pop().split("',", 1)
        comment = last_field.pop(0)
        # Filter spam comments
        if "spam" in ''.join(last_field):
            continue 
        # Put the valid comment into our field list
        field.append(comment)
        
        # Remove whitespace and unwanted characters
        field = [value.strip(" '") for value in field]
        
        try:
            # Check if line is entry of wp_comments table
            if "wp_comments" in field[marker]:
                # Check if comment is for the right blog post
                current_id = int(field[post_id])
                if  current_id == id:
                    # Return the useful field values
                    # as a dictionary
                    yield {
                        "author":   field[author],
                        "email":    field[email],
                        "url":      field[url],
                        "content":  field[content]
                    }
        except IndexError:
            pass # Just ignore invalid comments

if __name__ == "__main__":
    main()

That did the trick.

Why I love text files

2010-10-10T00:00:00-07:00

Text files are the single most important way we can communicate with computers. It’s no coincidence that they are also the most important way to communicate with other human beings. What we can achieve with text files is invaluable: Write it once and refer to it whenever you want to get the message across in the future. Write a program (it’s just text), save it and let the machine execute it whenever you like. Write another text file which contains the rules for the execution of your program and the computer runs your program exactly as you specified (cron files do that on Unix).

Text files can be structured in any way you can possibly imagine. Some flavours are JSON, Markdown and SVG. It’s all just text. There exist literally a billion of programs and algorithms to access, modify and distribute text files. You can write them with Emacs, print them on a terminal, pipe them through sed and send them via email to a friend who publishes them on the web. Because text files are so important we have good support for them on any computing system. On Unix everything is a file and HTML is just structured text. It’s a simple and powerful tool to make a contribution to society that outlasts our lifes.

I have a single text file in my mac dockbar which is called TODO.txt. I open it every day and after years of experimenting with different task management apps from simple command line tools to sophisticated online information storage systems I always came back to plain text files. And the explanation is simple: If mankind will still be around a thousand years from now, chances are that plain text files are one of the very few file formats that will still be readable by then¹.

They are an incremental part of how we can modify our environment without even leaving our desk. They have no overhead and can contain a single thought or the complete knowledge of our species. Distributing textual information is so vital for us that we permanently develop faster distribution networks – the fastest by now being the internet.

On the web you have instant access to a virtually endless amount of information and data distributed as plain text files. New web services made accessing the data even easier offering APIs and feeds. You can pull down the data from their servers and make statistics with a programming language of your choice. As you may have noticed, my affinity to text files partially comes from my programming background. As Matt Might correctly points out on his blog:

The continued dominance of the command line among experts is a testament to the power of linguistic abstraction: when it comes to computing, a word is worth a thousand pictures.

Whenever you like a text on the web just link to it and create a wonderful chain of ideas. Want to read it later or recommend it to a friend? Just share the text or print it on paper. The fact that we all take such things for granted is a testament for the power of text files and their importance for the information age.

Links

Unix Text Processing (PDF)

[EDIT] I admit that this paragraph is a bit misleading. I would be happy if my todo file was so important that it has to be readable in a thousand years from now but that’s not the case. I just like to be sure that I can access it without any special (proprietary) software whenever I want. My TODO.txt is just a personal, transient, disposable storage system (cat TODO.txt | wc -l prints 174 at the moment…).
↩

What about Microsoft?

2010-10-09T00:00:00-07:00

Roughly ten years ago, on October 29, 2001, Bill Gates shared a precise picture of the future of computing:

In the Digital Decade, you’ll no longer think of the PC as a tool you use only to carry out specific tasks it will become something you come to rely on all the time. The power of the PC will be as ubiquitous and reliable as electricity, and vastly more useful than any single device we use today.

While previously underestimating the web as a hype, Gates predicted magic things like the iPod, the iPhone and the Blackberry to soon be part of our lives. These are all mobile devices you carry around without ever noticing the embedded computer. You use them just like any other everyday object. Their very nature hides most of the underlying technical details (the operating system) from the user and has a strong focus on apps and content. They just work.

Monopoly on the desktop

In the early 2000s, smart-phones and mass appeal of the web were the logical next steps and Microsoft had every reason to be excited about that. They had record breaking revenues and absolute market domination in both operating systems and office products. While the quality of their products has always been a reason for debates they were brilliant when it came down to business: Create decent software, integrate it with other company products and sign persistent contracts with the distributors. That’s also how they won the Browser Wars against Netscape making Internet Explorer the de facto gate to web content. For the last 15 years the company grew from a tiny software startup to a billion dollar global player — and they wanted to be darn sure it stayed that way.

Harsh beginnings

When the net really started to gain momentum Microsoft was a bit late to the party like a cumbersome tech guy who’s watching all the hipsters gettin’ the phone numbers. To tackle the new rivals Microsoft started to mimic the behavior of the already successful guys.

As a reaction on Netscapes JavaScript, the Internet Explorer promoted ActiveX, a proprietary browser plugin that allowed Microsoft to run their own languages JScript and VBScript on every website to offer loyal Windows-Developers a simple development platform for interactive websites. Because of some major security flaws and the fact that it was a Windows-only product, ActiveX soon had a bad reputation amongst developers. Furthermore Microsoft extended the HTML and CSS standards with vendor specific extensions like filters and the marquee element. While other vendors like Mozilla or Opera did the same, the extensions from Microsoft were not HTML and CSS 2.1 compliant, provoking syntax validation errors and much anger.

The platform

The central concept has always been to build an exclusive platform. An area of total control (much like Apple recently). Microsoft was never ashamed to borrow a good idea from its rivals and sell a (locked down) version of its own. They have a track record in creating new document types and protocols (often times inferior to open standards) in order to strengthen the platform. While it seems unethical and has been the bone of contention for quite a few antitrust cases, it’s totally understandable from a companies’ point of view. By bundling Internet Explorer with every copy of Windows, they made it very hard for Netscape to compete, eventually pushing them out of the market.

Playing catch-up

In parallel they worked on a web portal called MSN (Microsoft Network) — just like AOL. They later built their own search engine — to rival Yahoo and created a proprietary chat application named Microsoft Messenger that is much like the successful ICQ. The list goes on and on. Whatever got their attention, they wanted a piece of the cake.

But as hard as Microsoft tried, the new startups had the fresher ideas, the smarter apps, the bigger user-base. Microsoft suddenly was in the role of the underdog. They tried to fight on many fronts at once while constantly failing to use their own creativity to outsmart the antagonists. One piece in the puzzle was always missing: innovation. While the technology was quite mature, some solutions still looked like dull, inferior, blatant copies of successful websites and web-apps that missed the final polish. This mentality lead to many setbacks and dead ends: Their social network Wallop was a stillbirth, just like MSN Music. More recently the company located in Redmond shutdown Soapbox and launched MSN Movies to compete with YouTube and other streaming sites but at the moment the site struggles to gain traction.

A new development approach

Obviously the web has a lot of characteristics that make Microsoft nervous: Open standards, rapid distribution of software and content, free alternatives to established desktop programs. Part of their initial resentment might come from the fear that the Windows platform – the operating system including all third-party vendors – might rapidly become irrelevant. They would have to start anew. So their strategy was (and still seems to be) to make the web more like an extension to their platform, just another part of the Microsoft universe. But their competitive advantage on the desktop is negligible on the internet. Some might even say Microsoft never really grokked the web.

A solution might be to look outside the box and support open standards (like OAuth) and show more commitment to free software. Much like Google which contributes to many important open source projects (Linux and Python) or Facebook which built a free PHP compiler called HipHop, Microsoft could be an active part of the community instead of constantly infiltrating it.

But what are the advantages for Microsoft to rapidly change the development process and give up their precious proprietary platform to create an open environment for the web and mobile devices? This can increase the development speed of large systems and lower the costs, as demonstrated by Google with the open Android platform or Apples WebKit which is used on many browsers (not just their own Safari).

As a site note it’s interesting to see how Apple gained popularity within the last ten years by gradually moving all their services to the cloud and offering a seamless interface on all their mobile devices. By recruiting droves of loyal developers for iOS development they now have the platform that Microsoft urgently needs. They have made the jump into the web by increasingly blurring the line between local apps and the web.

Keep in mind that success is never guaranteed. Google stopped Wave and Apple had a false start with Ping. But because of the agile development process and the wide variety of innovative products, these roadblocks aren’t fatal but often times vital. Microsoft needed 14 years to bring Internet Explorer to version 6, Google needed just two for Chrome (and it’s already faster than IE8…). Currently Google Instant makes Bing! look like a rusty piece of metal. Sure all that doesn’t mean Microsoft will disappear any time soon but without innovation any company is doomed to drift into insignificance.

An open future

But maybe Microsoft is already working on a more open policy: With their new Internet Explorer 9, Microsoft still has a strong focus on proprietary web technologies like Direct2D and Silverlight but they also pass the Acid3 test for standard compliance and support the new HTML5 canvas and video elements. They have the money and the manpower to shape the next big thing but it will be a hard piece of work and it takes a step back to see that the web is all about open standards and collaboration.

Exporting posts from posterous

2010-09-17T00:00:00-07:00

Posterous is awesome for great looking blogs without the hassle. It was great to have it for posting my thoughts when my old blog was all messed up. Now that I’ve redesigned it I want to host all my old posts on my new blog, too. Therefore I needed to export them somehow. It’s a bit strange that the guys from posterous don’t offer an export button on the configuration page but they help you out with a nice RESTful API that can be used with just about any language.

Here is how a response to a query looks like:

<post> 
  <url>http://post.ly/NWNC</url> 
  <link>http://matthiasendler.posterous.com/.../link> 
  <title>Overkill: Java as a first programming language</title> 
  <id>11579710</id> 
  <body> 
    <![CDATA[><p>I recently...]]>
  </body> 
  <date>Fri, 12 Feb 2010 08:12:00 -0800</date> 
  <views>505</views> 
  <private>false</private> 
  <author>Matthias Endler</author> 
  <authorpic>http://files.posterous.com/.../unix_thumb.jpg</authorpic> 
  <commentsenabled>true</commentsenabled> 
  <commentsCount>0</commentsCount> 
</post>

Grab your posts

So off we go with this little PHP snippet that gives you ten of my latest blog posts:

<?php

  // Posterous PHP export. Requires PHP 5.1.0 or newer
  // Documentation on http://posterous.com/api/reading

  // Set timezone for posterous blog
  date_default_timezone_set('America/Los_Angeles');

  // Query options
  $site_id = "";                // Either use id of the site...
  $hostname = "matthiasendler"; // ...or a posterous subdomain.
  $num_posts = 10;                  // Number of posts to read.
  $page = 1;                              // Get specific page.
  $tag = "";                     // Grab posts with these tags.

  // Create query string
  if ($site_id) $query = "site_id="  . $site_id;
  else          $query = "hostname=" . $hostname;

  if ($num_posts) $query .= "&num_posts=" . $num_posts;
  if ($page)      $query .= "&page="      . $page;
  if ($tag)       $query .= "&tag="       . $tag;

  // Request
  $url = "http://posterous.com/api/readposts?" . $query;
  $xml = simplexml_load_file($url);
  $post_count = count($xml->post);

  // Cumbersome way to walk post entries. 
  // See php.net/manual/de/function.simplexml-load-file.php#86471
  for($i = 0; $i < $post_count; $i++) { 
    $post = $xml->post[$i]; 
    // Now we can use $post->title, $post->date, $post->body...
    write_markdown($post, "post");
  }

?>

Put it into jekyll

I wanted to import the posts into jekyll so I used the function write_markdown() to do so.

<?php

/**
 * Formats a posterous post as markdown to use it with jekyll
 * array  $post   The post that gets processed
 * string $layout The liquid template to use 
 */
 function write_markdown($post, $layout) {
	// Extract relevant data from $post
	$date  = date("Y-m-d", strtotime($post->date));
	
	// Create filename
	$extension = ".mkdn";
	$filename = $date . "-" . strtolower($post->title);
	
	// Remove whitespace and special characters from filename
	$patterns = array('/\s/', '/:/','/\(/','/\)/', '/]/', 
	'/\[/', '/!/', '/\?/', '/\+/', '/=/','/\./', '/\"/', '/,/');
	$filename = preg_replace($patterns, '-', $filename);
	
	// Replace multiple dashes with single dash
	$filename = preg_replace('/--+/', '-', $filename);
	
	// Remove possible dash at the end of filename
	$filename = trim($filename, '-');
	
	// We're good to go. Create a new markdown file
	$jekyll_post = fopen($filename . $extension, 'w');
	if ($jekyll_post) {
		// Jekyll requires a yaml front matter
		$yaml = "---\n" .
			"layout: " . $layout       . "\n" .
			"title:  " . $post->title  . "\n" .
			"---\n";
				
		// Write front matter, post data and close file
		fwrite($jekyll_post, $yaml);
		fwrite($jekyll_post, $post->body);
		fclose($jekyll_post);
	}
 }

?>

It’s generally a good idea to use an HTML to Markdown converter for the post body. There is a nice one for PHP called Markdownify and a python script named html2text.

Download

You can download the whole script here. Simply execute it by typing php grab.php.

Download the package.

Errata

Thanks to Algert Sula for pointing out an error in the above script.

Sorting speed of STL vectors and lists

2010-02-12T00:00:00-08:00

This snippet is an addition to my previous article about sorting STL containers. I’ve tested the sorting speed of std::vector vs. std::list on my MacBook. As we can see by the results below, vectors are much faster than lists when it comes to sorting (almost 8 times faster on my machine).

Thx to Dean H. for commenting.

#include <algorithm>   // Needed for sort() method
#include <vector>      // STL vector class
#include <list>        // STL list class
#include <iostream>    // Needed for cout,endl
#include <sys/time.h>  // Needed for rand,timeval
using namespace std;   // Save us some typing

bool comp(const int& num1, const int& num2)
{
   return num1 > num2;
}


// Prints the difference between to time values to std::out
void printDiff(const timeval& end, const timeval& start)
{
    timeval diff;
    diff.tv_sec = end.tv_sec  - start.tv_sec;
    diff.tv_usec= end.tv_usec - start.tv_usec;
    if (diff.tv_usec < 0) {
        diff.tv_sec--;
        diff.tv_usec += 1000000;
    }
    cout << "Elapsed time: " << diff.tv_sec << "seconds, "
         << diff.tv_usec << " microseconds." << endl;
}

int main()
{
    vector<int> v;              // Vector containing integers
    vector<int>::iterator vIt;  // Iterator navigating through vector

    list<int> l;                // List containing integers
    list<int>::iterator lIt;    // Iterator navigating through list


    // Insert a 100 million integer values into both containers
    srand( time(NULL) ); // Initialize random number generator
    generate_n (back_inserter(v), 100000000, rand);
    generate_n (back_inserter(l), 100000000, rand);

    // Measure sorting time
    timeval start, end;

    // Sort vector
    gettimeofday(&start, 0);
    sort(v.begin(), v.end(), comp);
    gettimeofday(&end, 0);
    cout << "VECTOR: ";
    printDiff(end, start);

    // Sort list
    gettimeofday(&start, 0);
    l.sort(comp);
    gettimeofday(&end, 0);
    cout << "LIST: ";
    printDiff(end, start);

    /* Output:
     * VECTOR: Elapsed time: 99seconds, 566816 microseconds.
     * LIST:  Elapsed time: 770seconds, 216836 microseconds
     */
    return 0;
}

Overkill: Java as a first programming language

2010-02-12T00:00:00-08:00

I recently talked to a student in my neighborhood about his first programming experiences. They started learning Java at school and it soon turned out to be horrible.

A lot of us learned coding in languages like BASIC or Pascal. There was no object orientation, no sophisticated file I/O and almost no modularization…and it was great. In BASIC you could just write

PRINT "HELLO WORLD"

and you were done. This was actually a running program solving a basic and reoccurring problem: Output some text on a screen.

If you wanted to do the same thing in Java you simply write:

public class Main { 
  public static void main (String[] args) {
    System.out.println("Hello, world!"); 
  }
}

Do you see how much knowledge about programming you must have to achieve the easiest task one could think of? Describing the program to a novice programmer may sound like this:

Create a Main class containing a main-method returning void expecting a string array as single argument using the println method of the out object of class PrintStream passing your text as single argument.

— please just don’t forget your brackets. This way your first programming hours are guaranteed to be great fun.

OK. So what are the alternatives? I admit that nobody wants to write BASIC anymore because of its lack of a sophisticated standard library for graphics (Java doesn’t have one either) and its weak scalability. The language has to be simple and clean. It should be fast enough for numerical tasks but not as wordy as the rigid C-type bracket languages (sorry C++ guys). It should have a smooth learning-curve and provide direct feedback (compiled languages often suck at that point). It should encourage clean code and reward best practices. One language that provides all that is Python.

And Python has even more: hundreds of libraries that help you with almost everything, good integration into common IDEs (PyDev in Eclipse, IDLE…), a precise and elegant syntax.

Here is our program from above written in Python:

print("Hello World")

There’s no need to know about object orientation, scopes and function arguments at this point. No house-holding or book-keeping. Yes, it’s an interpreted language but that’s not a deal breaker for beginners.

If you aren’t convinced yet, printing and formatting text output in Java is relatively easy for an advanced programmer but the really ugly stuff begins with file input:

import java.io.BufferedReader; 
import java.io.FileNotFoundException; 
import java.io.FileReader; 
import java.io.IOException; 
 
public class fileIO { 
    public static void main(String[] args) { 
        String filename = "test.txt", line; 
        try { 
            BufferedReader myFile = 
                new BufferedReader(new FileReader(filename)); 
            
            while ( ( line = myFile.readLine()) != null) { 
                System.out.println(line);
            } 
        } catch (FileNotFoundException e) { 
            e.printStackTrace(); 
        } catch (IOException e) { 
            e.printStackTrace(); 
        } 
    } 
}

I hear you say: “Dude, file I/O is pretty complex. It’s just the way it is”. That’s true… internally . But the user should get an easy interface. Python shows how it’s done:

file = open("test.txt")
text = file.read()
print(text);

The code goes hand in hand with the natural understanding of how the process works: “The computer opens a file, reads it and prints it”. Even a five year old kid can understand that. Nobody would start to explain: “Before you can read a file you need a BufferedReader that works on a FileReader…” even if this is exactly how it works internally. You want to explain the big picture at first. The elementary principles of teaching a computer how to do useful stuff. Otherwise you will start frustrating beginners and fool them into thinking that they are not bright enough for programming. Programming is fun and starting with it is the most important step. So don’t spoil that experience with layers of unneeded abstraction.

Sorry Apple, that iPad sucks!

2010-01-27T00:00:00-08:00

When Steve Jobs held it in his hands my first thought was: “That thing is fat and it somehow looks like a giant iPod”. Just like somebody lost his Apple phone on a bus and a million people sat on it. Obviously it looks like we all expected. A black glossy surface with aluminium highlights here and there. On the bottom is that compulsive “help me!” button and that’s about it. Oh! And it’s thin. Yay!

From a hardware point of few nothing to fall in love with. It can’t cope with the sophisticated elegance of the iPod that takes full advantage of its size with its zoom and pan functions and it doesn’t sport the versatility of a full-fledged MacBook Pro with all its ports and gizmos.

Unfortunately the software side is even more unimpressive. That iPhone OS with its tiny icons, its useless dockbar, its single-tasking core looks lost and abused on this 10 inch display. It fits well for a phone where display size is limited but on that fancy new “web device” a lot of potential is wasted. Why not design a new web-oriented interface with a calender and facebook as a desktop. With recommended YouTube videos and new mails at your finger tips. Instead you don’t even get Flash. You can’t install Firefox. Eww.

Apple wants to keep its babies separate. Your iPhone for calls and navigation, your laptop for “hard stuff” aka “work” your iPod for media and your iPad for…umm…surfing? What you get is a hybrid device that does essentially nothing.For hard number crunching you’re better off with your desktop workstation or notebook. For music an iPod fits your needs. Chatting and phone calls are reserved to the iPhone. SMS too. On the other end your iPad does not even have a webcam.

So you’re left with that overpriced eBook-reader that includes a web browser and 14.000 apps that were designed for a 320x480 pixel screen. Oh and for eBooks we’ve got the Amazon Kindle which is cheaper and doesn’t hurt them eyes while reading because of that fancy E-Inc technology. Btw you get two days battery lifetime, too. Out of the box the iPad may be the most expensive book you’ll ever buy. It may soon go the way of the Macbook Air and the Apple TV. Nice idea but irrelevant.

Another tricky part is writing text on that device. Everything that’s longer than an URL is a pain on that glass surface. On top of that you need to put it on the table and bend over the device to use all your fingers for typing. Just hope the sun does not shine.

I don’t say this won’t become a huge success for Apple when there will be new apps and hackers to get their hands dirty on that device. A pirated version may do all sorts of crazy stuff but that’s illegal and you’ll loose your warranty. There is a long way to go for Apple. They just don’t understand that web also means open.

Howto sort a vector or a list in C++ using STL

2010-01-27T00:00:00-08:00

A little code snippet that people need very often:

/*
 * Howto sort a vector or a list in C++ using STL
 */

#include <algorithm>	// Needed for sort() method
#include <vector>	// STL vector class
#include <list>		// STL list class
#include <iostream>	// Needed for cout,endl
using namespace std;	// Save us some typing


/*
 * This is a comparison function. It can be used to tell sort()
 * how to order the elements in our container (the vector or list).
 * You can write a comparator for every data type (i.e. double, string...).
 */
bool comp(const int& num1, const int& num2) {
	return num1 > num2;
}
int main() {
	// SORTING WITH VECTORS //
	vector<int> v;
	// A vector containing integers
	vector<int>::iterator vIt;
	// A pointer to a vector element
	// Insert some values
	v.push_back(5);
	v.push_back(12);
	v.push_back(1);
	/*
     * The generic STL sort function uses three parameters:
     * v.begin()  Iterator pointing at the _beginning_ of the container
     * v.end()    Iterator pointing at the _end_ of it
     * comp       [Optional] A comparison function (see above)
     *
     * The above mentioned iterators must be random access iterators because
     * sort() takes advantage of clever tricks that require direct access to
     * all elements of the vector. This makes it really fast.
     * (Currently introsort is used with O(n*log n) even in worst case).
     *
     * Unfortunately calling sort like that does not look very object oriented.
     */
	sort(v.begin(), v.end(), comp);
	cout << "Vector: ";
	for (vIt = v.begin(); vIt != v.end(); vIt++) {
		cout << *vIt << " ";
		// Print current element to standard output
	}
	cout << endl;
	// SORTING WITH LISTS //
	list<int> l;
	// A list containing integers
	list<int>::iterator lIt;
	// A pointer to a list element
	// Insert some values
	l.push_back(5);
	l.push_back(12);
	l.push_back(1);
	/*
     * Here is the major difference between vectors and lists in general:
     * Vectors offer fast random access to every element
     * but inserting a new element at the beginning or in the middle is slow.
     * On the other hand inserting into a list is fast but searching for
     * a specific element is slow. Vectors behave much like an array
     * while lists only allow slow sequential access.
     * Therefore we need a different function to sort all elements that does
     * not need random access iterators.
     *
     * comp       [Optional] A comparison function (see above)
     *
     * Note that sort() is specific for the list and is implemented as a
     * member function of list<>. This is much more object orientated.
     */
	l.sort(comp);
	cout << "List: ";
	for (lIt = l.begin(); lIt != l.end(); lIt++) {
		cout << *lIt << " ";
	}
	cout << endl;
	/* Output:
     * Vector: 12 5 1
     * List: 12 5 1
     */
	return 0;
}

Web-Development with jQuery

2009-11-25T00:00:00-08:00

I’ve searched for some IDEs to create jQuery code more quickly. Hacking the files into a text-editor (like Textmate) and doing some “real- life” bug testing by simply opening Firefox and running the site is not very handy. So here is a list of alternatives I came up with. It’s just an overview loosely ordered by interest.

Aptana

Standalone and Eclipse plugin. OpenSource with code completion, syntax highlighting…

Komodo Edit

OSX, Linux, Windows. Open source.

Glimmer

Microsoft jQuery Dev. Environment. Has wizards for common tasks. Free.

FireBug

Firefox plugin for debugging. Actually not an IDE.

IxEdit

Edit within browser. WYSIWYG. Free public beta.

Spket IDE

JavaScript Toolkit. Eclipse Plugin.

JsLint

Online JavaScript parser and debugger.

CodeLobster

For PHP/HTML/CSS/JavaScript development. Trial.

Scintilla

GTK based professional IDE

Others include Visual Studio, NetBeans, Dreamweaver and another JQuery Eclipse plugin. Jaxer looks also interesting.

And finally here is a cool website with tons of material ajaxian.

Did I miss one?

Running legacy code

2009-11-08T00:00:00-08:00

This short article deals with a serious problem in software development. When switching to a new platform (for instance from Windows XP to Windows Vista/7) the programmers need to make sure that old bits of code run flawlessly. There are several ways to achieve this goal that will be discussed during the next paragraphs:

Porting the code

This is generally considered a hard path to follow. For non-trivial legacy code-blocks chances are high that they contain side-effects and hacks in order to make them work in different environments. Porting code means replacing parts of the program that use functions and methods that don’t exist anymore with new ones which make use of the modern libraries and routines of the new platform. The major advantages are maintainable software and sometimes faster running programs. But it may be needed to hack the new platform libraries in order to preserve the whole functionality of an old program. When changing an algorithm inside legacy code the ported version may become unstable. Thus there may be better ways of mainting obsolete code today.

Emulators

Emulators work much the same like porting the code. You replace old function calls with new ones in order to make everything work again.
However you don’t alter the old codebase itself (because you may not have the sourcecode available) but you create a new compatibility layer that “translates” the communication between the underlying operating system and software (our new platform) and our old software. Emulation can also be very fast and run stable for many years but writing an emulator can be even harder than porting the code because an educational guess may be needed to figure out how the program works internally. Additionaly the emulator itself may become obsolete in the future and might eventually be replaced by a new one.

Virtual machines

During the last years a new approach was rising up. The idea is simple: Don’t touch anything. Take the whole platform and copy it in order to run old software. The old software runs on top of the old operating system within a virtual machine that runs on the new platform.

From a sane software developers view this method is ridiculous. A lot of resources are wasted along the way. The system is busier switching contexts from an old platform to the new one and back than running the actual legacy program. However with cheap and capable hardware everywhere this idea gets more and more interesting. As Steve Atwood coined it:

Always try to spend your way out of a performance problem first by throwing faster hardware at it.¹

And he’s right. The Microsoft developers did the same on their new NT 6.0 platform (Vista, Windows 7, Windows Server 2008…): Windows XP is running inside a virtual machine. This way everything behaves just like one would run the software on the old system. And by optimizing the performance bottlenecks (input/output, context switches) one gets a fast and stable, easy to maintain product.

Every method has its major advantages and disadvantages. It’s on the devoloper to select the appropriate strategy.

Article on Coding Horror ↩

A short Review on ReactOS 0.3.3

2007-09-18T00:00:00-07:00

FreeWin95 has come a long way since it has been renamed in 1998 and is now called ReactOS.

The new version 0.3.3 of the Windows 2000 clone already has a lot to offer and has significantly evolved in almost all areas compared to the last release. The developers have made a lot of progress in the lower system area (i. e. the NT Kernel) as well as in the application segment. Let’s have a look at some new features!

Setup

Due to the fact that I’ve tested ReactOS using the pre-configured VMWare Virtual Appliance (VMWare Fusion Version 1.0 (51348) on a MacBook 1.83 GHz to be precise) I can’t say much about the installation process of the new version using “real” hardware. However I’ve tried to install it with the Install CD that can be downloaded on the project homepage. Unfortunately I wasn’t very successful. Somehow ReactOS was not able to allocate the free space on my virtual hard disk…The last time I used the Install-CD on an old 300 MHz Computer I simply wiped my old partition out, created a new one (fat32 only guys :), formatted it and started the Installer. I had no problems. On the other hand it can be dangerous to install ReactOS on the same disk as Windows or Linux. Maybe the Installer doesn’t care and assimilates all disk space (All your system are belong to us). But: Hey! MS does the same with all competitors, and maybe ReactOS simply aims to clone that approach ;) However the Virtual Appliance worked very smooth on my system. During the test I didn’t have much problems setting up and using the virtualized OS.

Bootup

When I fire up the machine a nice bootloader appears and you may select between the normal and the debugging mode. Pushing F8 gives you another menu where you can select more booting options - like a command prompt-only version or the infamous last good option. For the moment I simply waited eight seconds until the normal boot process started.

The developers have changed the bootup screen a bit by adapting the XP look and feel. Generally said the bootup process is very streamlined and offers a whole variety of options for the advanced user. But even the normal PEBCAC can easily reach the desktop interface. Presently the ReactOS kernel (which aims to be compatible with WinNT 5.2) offers about 25 to 30 percent of the functionality it will finally have in 1.0. As a result of the broad kernel redesign in 0.2 to 0.3 the current release faces a lot of regressions in some applications that were already useable in 0.2.

Software

ReactOS actually comes with a small amount of pre-installed software packages. Taking a closer look the team is working hard to offer a free replacement of commonly used Win Software. Naturally it has the Command Prompt as well as Regedit, Notepad and WordPad. It also features a scientific Calculator as well as the not-so-well-known Windows commandline classics like shutdown.exe, regsvr32.exe or telnet. Most of these basic programs were borrowed from the wine project. Together with the relatively stable kernel ReactOS already has a solid basis for further software development. Such Bluescreens are (really!) an exception: If you need a package that is not installed by default you can upgrade your system by installing software just as you would on Windows. There are many packages that run out of the box without further tweaking. Just look at the list of supported software on the interwebs! The newly integrated Downloading-Tool - simply called Download! - can help you to install compatible software with just one click. Because it’s work in progress there are not so many programs available. You can select between the categories Internet & Network, Development, Games & Fun and Tools. Some cool open source apps like Firefox, Thunderbird, Samba, OpenTTD, 7zip and μTorrent are already ready to use. In later versions it will be possible to update the list of preconfigured packages. That’s why I see the Download! feature as a promising invention.

Hardware acceleration

Despite the fact that 0.3.3 has all the necessary drivers on board to run on almost every machine without any need for configuration it may be quite interesting to see ReactOS on full speed using original hardware drivers. In my case I just needed to install the VMWare toolset that offers a major speed burst when installed. I’ve added a CD-Rom Drive to my virtual machine and clicked on Virtual Machine > Install VMWare Tools while ReactOS was powered on. To my surprise almost everything worked like a charm and I could install the VGA and the audio drivers to tune my system. After a reboot - my system was messed up. The VGA resolution was down to 320-something pixels which made using the OS an impossible task. I had no chance to adjust the resolution with the common Display Properties window nor did it work to use Regedit and adjust the screen values manually. My second thought was “No problem”. In case of an emergency we have the Debug Mode on board. Well, what can I say: It didn’t work. It has the same screen resolution as the normal mode. Maybe there is a workaround to fix it but I was too lazy to figure it out and so I simply reset my virtual machine (There you have it, bastard!). The free Windows clone would be quite a usable system for a normal user if some of the configuration tools were already integrated. There is no printing implementation and you can’t presently use USB or firewire devices. The latest release offers only rudimentary network support. That said the team has worked hard to offer accessibility features as well as localization adjustments. At the moment one can select between 90 (!) supported languages

Stability

ReactOS still is (and will be for a long time) an experimental System. It’s normal for an unstable pre-release of an alpha software to randomly freeze or hang or kill your machine or whatsoever. Don’t panic when you have to restart - it’s (presently) normal. Don’t expect ReactOS to run all your .exe files it is a long and painful procedure to reach this aim. That said ReactOS 0.3.3 is surprisingly stable compared to older versions. Yes it occasionally hangs and it takes a nap from time to time when all of your ram gets pwned. But don’t be afraid: Just feel free to hang around a bit dazzling with the software - actually having FUN using your computer. 0.3.3 is the first version that doesn’t almost certainly crash after two minutes of use or a wrong click in the system configuration menu (o_O). ReactOS has grown up from a pure hacker playground to a mature, promising open source alternative for proprietary M$ desktop products. Certainly you’ll find a bug and maybe want to help the developers. In this case just go to their site and fill out a bug report and you can say to yourself that you’ve just supported free software. The thing a good developer (and good engineer in general) likes most is when somebody says Thanks for all the great work! That’s what I also like to say to the ROS team. Keep up the good work! According to Wikipedia the next major release (i.e. 0.4) will offer basic USB support and a complete SMB interface (making it fully compatible with the Windows network subsystem). After that ReactOS will enter beta status and we will see all the fancy user-world stuff like NTFS support or DirectX 7-9 support. So there is a long way to go and a lot to explore!

Linux needs to evolve - now!

2007-01-04T00:00:00-08:00

It was a brilliant idea to create an operating system that is free for everyone to use and to edit. Thousands of hackers around the globe share the dream of open source information technology without monopolism. Everybody who is into computing has heard of Linux and lots of them are using it for daily work. Even inexperienced users try out Linux by testing distributions like Ubuntu, openSuse or Knoppix because they are easy to use and to install. But that’s not the complete truth.

Hardware woes

This so called “out-of-the-box” experience can instead be very tricky and much to complicated for a normal user. When you insert the Ubuntu LiveCD you will be welcomed by a smart boot menu that lets you try out all the functionalities of a full-fledged Distribution without installing.

The window manager has a fresh, clean-looking aqua-style interface. You click on install and almost everything is pre-configured and runs automatically. In most cases a distribution like Ubuntu auto-detects your hardware and installs the right modules. Hopefully. Sometimes it happens though that your printer is not working or your wireless connection was not recognized. This is mostly the case when you have shiny new hardware that linux hackers were not able to test yet. This can be a brand new inkjet printer (cups and gimp-print have a long way to go) or the high-end graphics card.

Hacker perception

Not a problem you might think. Look for a new development driver on Google, do a quick apt-get, set-up your network connection and off you go! BUT explain that to Aunt Tillie sitting in front of her PC thinking that a Wlan card is a trading card her six year old nephew Gregory wants for Christmas. I don’t have to add that she never opened a PC because she simply doesn’t bother what’s in there. You might say that it’s also hard to configure your network a Windows PC. Well maybe it is - but it’s already configured when you buy the PC - and that makes a big difference. Installing new software packages can be painful for normal users. Have you ever tried to explain to a non-geek why it’s handy to use a terminal for daily work? At best you get an answer like ”I’ve used DOS and that was a piece of crap” or maybe he just says “You are still sitting in front of a black text-mode only window entering cryptical commands to configure your system? This is not 1969!”. Many people don’t understand why typing chmod is smarter than clicking through ten windows opening the “properties” submenu with a right mouse click on a file and finally adjusting the access rights. Why should somebody use ‘dmesg | tail’ to get some system information. The problem is: When somebody refuses to learn those shell-commands they won’t understand 90 percent of the system. They can’t see what went wrong when there is some internal dependency error. Linux will only be easy when you understand what you are doing.

Software woes

Even if you know how to work with a command line you can’t be safe from so-called ”dependency hell”. You want to install a new office suite called “OfficeSuite” and download the packages directly from the developers site. But sadly it won’t run because you don’t have those three other packages installed which are all needed by “OfficeSuite”. OK. So you install those packages as well but they depend on three other libraries that you don’t have as well. As a result you end up downloading and installing six packages in order to run “OfficeSuite” - now that’s what I call productivity from the very beginning. The big problem behind this mess is the fact that there are no real standards in the Linux world. There are thousands of different flavors and to some extent this is great and it’s exactly what free software has to look like but to get more traction we urgently need better standards to focus developing power. Don’t get me wrong: There must be different package formats (like rpm, deb, tgz…) or various window managers (KDE, Gnome, fluxbox, icewm…) but they all must have a mutual basis to build up on. As we have seen the biggest problem is usability. Other Unix-like systems such as Mac OS X show how it can be done. Common guys! We can do better than that!

Why it matters

Some of the brightest Linux advocates don’t realize that non- techies want a system where they don’t need to know anything about bash, emacs or vi to successfully configure hardware drivers and network sharing (samba…urgh). Those who say “Screw it. If Linux is too hard for you, just don’t use it” are misguided. It is vital for open technologies to reach a critical mass and receive widespread adoption. Otherwise big companies like AMD and NVidia and EA have little interest to invest development time into open source projects.

Howto switch from Linux to Windows - a users experience

2006-09-20T00:00:00-07:00

I’ve read articles like “Switching from Windows to Linux in five easy steps” or “How changing your operating system can be fun with Linux” or “Why you should give Linux a chance” for five years now but there are always people who say Linux is difficult to learn and Windows has much easier install routines and so on. But what if the first OS you’ve ever used wasn’t MS Dos or Win95 but some kind of Unix-based free system like BSD or Linux?

This is a text about a guy who has always used Linux and learned working with it when he was a young man when he had to use it at work. Imagine Linux is the standard OS and Microsoft is a small company offering so called MS Windows XP. This text is meant to be satiric so don’t take that too seriously! It is a fictional story that ought to be fun. So repeat after me: I will not take it seriously or otherwise not read it at all.

My first steps with Windows XP

I’ve heard of a new Operating System called Microsoft Windows the other day and wanted to give it a try because it is said to be easy to use and intuitive. Unfortunately the author offers no live-cd to test everything before I have to install. Instead of downloading it from the Internet as I always do I had to go to the computer store and buy it for a price of $150. There is also a so called Professional Edition but it would cost even more. The first thing when I came home was to open the box, put the included CD into my DVD-Drive and start my PC. The install screen consisted of a simple blue background and an easy-to-use menu. I wanted to backup my data at first so I tried to open a virtual terminal but the normal CTRL-ALT-F1 didn’t work (WTF?). I had to reboot, make a copy and restart the installation all over again. The menu presented an option called “Install Windows XP” and I selected it.

Then there was a strange message that said “End-User License Agreement” and that I have to accept it in order to continue. I have never heard about something like this on Linux so I read most of the long text. I am not allowed to distribute this product and I may not make copies for my friends. This was somehow the extreme opposite of a license I’ve previously heard of that is called Gnu GPL. After I accepted it with F8 the installation continued. Windows didn’t recognize that I had ext3 and reiserfs partitions on my system and told me that it had found an “unknown filesystem”. I always thought that they were standard but maybe I was wrong. There was an option to erase the whole harddisk and install WinXP and I selected it. The next curious thing was that Windows doesn’t create a separate partition for my precious data but only one big system partition instead. “Maybe Windows has a highly secure file-system that makes automatic backups in case of data loss” I thought. Because I have a relatively big disk I wasn’t able to select Fat32 and had to cope with NTFS. It took a while to format my disk.

Unfortunately I wasn’t allowed to select any packages from a list as I always do on Linux - Windows selected everything alone instead. The only adjustments I could make were entering the correct system-time, my username and something called registration key. This is a 25-digit number I had to enter manually before setup continued. After another reboot Windows was finally up and ready - well almost. Windows wanted to be activated at first. Within this procedure Microsoft tries to access the web to verify all registration data. No way! I canceled and Windows told me that I have a 30-days trial until I must register. That sucks!

The new user interface appeared and I began to forget all the initial difficulties. It was a bit strange that there was no prompt to enter a default user for daily work. At the moment I was in administrator mode and had root access. That made me feel a bit uncomfortable because my little sister would have the same rights on that system. There also was no option to adjust file rights (something like chmod) manually and this could be another possible threat on a workstation. I’ve looked for system documentation material to start adjust my new system just as I want it but I couldn’t find anything usable (F1 is really no help). Suddenly an ugly yellow pop up appeared in the bottom right corner. It said something like “New device found.” A window in the middle of my screen told me that I should insert the device driver CD for my printer and click on “Continue”. At first I didn’t know what that meant but suddenly I knew that a device driver was something like a module. I remember getting a CD together with my printer and I found it in a box on my attic. I clicked on “Continue”. My new system warned me that the device driver has not passed the hardware compatibility test and that I should try to find another driver. I was a bit afraid to click on “Continue” but finally I brought myself to do it and - everything ran flawlessly. I was a bit irritated why this stupid message annoyed me but I didn’t have time to fuss. Afterwards I installed all the other hardware and was ready for surfing.

There was no firewall or anti-virus software installed and I felt a bit insecure when I first entered the web. I surfed for maybe two minutes before I received a message from “Al Dick” who offered a six-year-ration of Viagra. Despite the fact that I don’t need such things (really!) it was unbelievably annoying. I found a howto on the net to turn off such messages. The Internet Explorer doesn’t have a popup filter and seems to attract ads and after fifteen minutes my taskbar looked like a battlefield full of spam. I had to close every window manually. My former Browser Firefox had tab-functionality included but IE seems to have own browsing rules and standards. But that was just the beginning: When I wanted to chat with my friends I found out that Windows Messenger didn’t support ICQ, AIM, TOM and Jabber. I was forced to create a so called MSN-Account to continue but I refused. When I wanted to watch a newly bought DVD I got from France Windows Media Player told me that I may not use my DVD on my PC because of a so called “region code”. I was a bit confused about all this and asked a friend what to do. He told me that there is data transfer from a PC running Windows directly to Microsoft in order to check my hardware specifications and to make a genuine check of my software. I was shocked. I was paying a lot of money to own a stable, secure system that respects my privacy and not a buggy, talkative piece of crap open to others like a barn door. I wanted to take a look at the source code to see if those recriminations were right but I couldn’t because there was no source included! I don’t want to talk about the so called “file hierarchy” of XP where the systems config files can be accessed via C:\Windows\system instead of /etc. You don’t want to know what defragmentation is (there’s no need for it on a linux box), I won’t speak about the lousy terminal called “command” and why you should never ever open email attachments with Outlook Express.

After the system crashed when I wanted to burn a CD with the skinny burning tool included while doing some spreadsheet analysis with Microsoft’s miserable OpenOffice alternative called “Office XP” that cost me another $119 (I got it a bit cheaper because I’m a student) I put everything back into the nice green box and took it back to my trader. At the same day I installed Linux again giving a review on a half-baked, single-user operating system called Windows XP that may be ready for desktop use in about five years. Until then I enjoy freedom with BSD, OpenSolaris and Linux.

Linux games

2006-08-30T00:00:00-07:00

Here are some linux floss games I’ve tried (ordered by personal rating):

Mania Drive
(9/10)

I’m totally fascinated. It is almost too simple but it makes so much fun (especially when you play with a friend) that you can’t stop until you’ve played all levels. Mania Drive can be understood as a tribute to TrackMania - a freeware game (well partly). I actually did a level myself than can soon be played via internet. It is called “Super Clio Brothers 3 !” and it beats SCB 1 and 2 in its craziness. Try it! Here is the official Mania Drive homepage and here is the "Super Clio Brothers 3 !" track. Feel free to share this link.

Pingus
(9/10)

You know Lemmings. This game is a clone but with much better graphics, sound and level design. It uses the SDL library and is therefore highly portable. It runs on Windoze and Linux (and maybe OS X) but I haven’t tried it yet. Give it a try - you won’t be disappointed. Here is the offical homepage

Super Tux
(9/10)

Can’t help but I have to give another nine out of ten points. This time Tux follows Super Marios footsteps and jumps through (partly very tough) levels. Here is the project page.

Klotski
(9/10)

how I love and hate it. The only thing to do is to move some bricks to finally get one out of a box. Sounds easy? Try!

Neverball
(8/10)

Neverball is a game where you have to control a little ball over various platforms and mechanisms without letting it fall down. This can be harder than you think! Neverball has comic-like graphics and nice shadows and reflections. There is another game that can run on older PCs and doesn’t have bombastic graphic effects but makes the same fun: Trackballs. Download Neverball here.

Nexuiz
(7/10)

This is a shooter with nice graphics and balanced gameplay. I often play shooters and there are a lot of them out there available for free. It’s a pity that there are no bots in the game so it can only be played on lan parties or via internet. Look here to download.

Cube
(7/10)

And again we have a shooter here. This time it is a fast and dynamic shooter that has its focus on lan gamers and simple gameplay. It has the highest framerate of those three mentioned and it is optimized for speed. One may like it or not. Download here! There is also a new engine still in developement: Sauerbraten/Cube2.

Warzone 2100
(7/10)

Strategy game like Starcraft or Command and Conquer. It is in 3d and runs fast on my machine. The battles can really make fun and the AI is sometimes very clever. It has previously been developed by Pumpkin Studios and published by Eidos before it was GPLed. Look here.

Tremolous
(6/10)

Another shooter that wants to bring in some new ideas into the genre. Don’t expect to much because it is in an early stage. You can already play via network but there should be single player mode. It is based on Quake3 and it was the first game using its source code since it now is published under GnuGPL. Here is the offical page.

Tools

Are You A Programmer?

On hard work

Dedication

Direction

Devotion

An open letter to Apple

My experiences with Feedback Roulette

Digg – a case study for the social web

Taking the wrong way

Up next

Lessons learned

Importing old Wordpress backups into Intense Debate

Why I love text files

Links

What about Microsoft?

Monopoly on the desktop

Harsh beginnings

The platform

Playing catch-up

A new development approach

An open future

Links

Exporting posts from posterous

Grab your posts

Put it into jekyll

Download

Errata

Sorting speed of STL vectors and lists

Overkill: Java as a first programming language

Sorry Apple, that iPad sucks!

Howto sort a vector or a list in C++ using STL

Web-Development with jQuery

Running legacy code

Porting the code

Emulators

Virtual machines

A short Review on ReactOS 0.3.3

Setup

Bootup

Software

Hardware acceleration

Stability

Linux needs to evolve - now!

Hardware woes

Hacker perception

Software woes

Why it matters

Howto switch from Linux to Windows - a users experience

My first steps with Windows XP

Linux games

Mania Drive (9/10)

Pingus (9/10)

Super Tux (9/10)

Klotski (9/10)

Neverball (8/10)

Nexuiz (7/10)

Cube (7/10)

Warzone 2100 (7/10)

Tremolous (6/10)

Mania Drive
(9/10)

Pingus
(9/10)

Super Tux
(9/10)

Klotski
(9/10)

Neverball
(8/10)

Nexuiz
(7/10)

Cube
(7/10)

Warzone 2100
(7/10)

Tremolous
(6/10)