PHP is not Java!

Now to something completely different:

(might be trivial for Excel Professionals, so forgive me that i am so naive)

My management asked me some days ago for the cashflow of my current project. I know basically how this needs to be calculated: You define a period of time, take all money that comes in and substract all money that goes out. You do this for every period and then you have a cashflow per period. Then you want to calculate an accummulated cash flow so you see, if you need to put money into a project or if it brings in all that it costs throughout the runtime. So far, so good. But how to do it automatically. What do i have?

Incoming invoices (those which are already there and the forecasted ones) with due dates:

10,000.-- EUR on Oct 03 from XYZ

5,000.-- EUR on Oct 23 from BlaBla

20,500.-- EUR on Nov 05 from ABC

etc.

Then the invoices we send out with their respective due dates:

100,000.-- EUR on Oct 01

50,000.-- EUR on Nov 01

... both are lists on separate pages in my excel workbook. Now i need to split the lines by month and add up the amount of a certain month. How to do this? Pivot Tables are cool for such things but not flexible enough. After some research i found a great Excel function for this called "SUMPRODUCT" (in german: PRODUKTSUMME).

Assuming, the due date of the incoming invoices is in column E of a sheet called "invoices in" and the according amounts are in column D, you would calculate the incoming amount for a given month like this:

=SUMPRODUCT((MONTH('invoices in'!$E2:$E200)=10) * (YEAR('invoices in'!$E1:$E200)=2009) * 'invoices in'!$D2:$D200)

This will give you the sum of all amounts (column D) where the month of the due date is 10 and the year is 2009. You can also replace the 10 by a reference to another date field (Oct 1 2009) (in cell C5 for example) and then make a table with date fields in the header for each month and below this you just fill this formula in a slightly modified form:

=SUMPRODUCT((MONTH('invoices in'!$E2:$E200)=MONTH(C5) * (YEAR('invoices in'!$E1:$E200)=YEAR(C5)) * 'invoices in'!$D2:$D200)

This way you can count all incoming vs. all outgoing cash. All you then need to do is to subtract and there is your dynamic cash flow. What i added then was a nice diagram of course. Works well and looks good

After i re-entered the open source version management area again (used Perforce for years and i know why ) i now work in a project with a lot of external partners and now we use Subversion.

While most others have no problems with update performance, it is horrible for me. Our repository consists of 8 GB and a couple of thousand files and our current revision number is 18644. So nothing specatcular (with perforce we were in the 1 Million change lists area).

Now clicking on "UPDATE" on my nice Vista notebook takes up to 5 minutes before the first files are actually transferred. While there was only little network traffic, HDD is working like mad. I could not explain that and so i searched for reasons online.

First i suspected Tortoise to be the bad guy. But after i had de-installed it and used command line SVN i learned that it is not the case (Tortoise has its oddnesses, but my bad update performance was obviously not linked to it). Also, our server is not the problem. It works fast for other clients. Eventually i found this blog post, that gives a hint:

http://svn.haxx.se/users/archive-2009-09/0879.shtml

Obviously SVN is using a lot of small files to lock the local copy. And under windows this kills performance especially on notebook-hdds. The statement seems to be: Wait for 1.7 and then the locking mechanism for SVN will be replaced by something more optimized for Windows Platform.

In the meantime i will have a look for acceleration for handling of a lot small writes for NTFS...

Update: I have now enabled "write caching" and "performance optimization" for the harddrive i use with SVN. Unfortunately that still does not do it. SVN UP still takes 5 minutes. What i see is 100+ page faults per second, but no significant network I/O and also not too much HDD throughput (1 MB/sec).

After 2 weeks in lovely Alghero i am back in the office. It is not as warm here and the sea is missing.

But the internet is much faster. And no roaming. So be prepared, Mafia Wars players! I am back in the game

Ever wondered what Google Maps was for? Of course for fun! The old idea of Monopoli now goes web and world wide. Since September 9, you can buy any street worldwide and build houses in a massive multiplayer Monopoli. What a cool idea! http://www.monopolycitystreets.com/game.html#de

Interested in an introduction to namespaces in PHP 5.3?
Zend has a webinar led by Stas regarding this topic tomorrow (September 15, 9 am PDT. According to my Outlook calender it would be 18:00 german time)
Javascript editor currently broken, therefore copy/paste this link for more information:
http://www.zend.com/en/company/news/event/webinar-namespaces-in-the-wild-with-php

On the weekend i attended the #phpunconf in Hamburg. We had a lot of very interesting discussions there with Michael Mayer, Johann-Peter Hartmann and Cornelius Weiss about PHP in the Javascript world. In these discussions we found that there are basically two approaches discussed at the moment:

Server side frameworks integrate JS widgets into their libraries now. They try to generate JS code from templates on the server side to control the user interface. This way, forms can be enritched with some special input fields that can provide nicer user interface (for example date-pickers, grids, tab controls, etc.). The widget structure that JS libraries like dojo or JQuery provide today seem to be targeted on this kind of usage. It appears to be a solution for sites with low user interface requirmements or for a transition period. Would that be RIA 1.0?

However, a lot of applications will need much more. Users demand top-level interactivity also from web applications these days. That introduces a different approach to javascript (RIA 2.0?). The server only sends one page and a bunch of Javascript to the client. From then on, the application flow is controlled by the client (JS) code. The server only supplies data here and there and executes transactions triggered by the JS application. For this kind of application, more code structuring is needed on JS side. Fortunately, there are already frameworks for this approach. JavascriptMVC is one of them. It contains a special JS-adapted implementation of the MVC pattern and is completely event driven. It even has a template engine implemented in JS in order to separate code from html when the JS app renders data from the server into html fragments to insert in the DOM structure of the page.
With this kind of structure elements, the Javascript code can be structured in a way that even big javascript applications can be built, maintained and automatically tested without losing overview. For the application we are currently building, this appears the only feasible approach.
Especially functional testing with tools like Selenium becomes fundamentally critical when feature rich JS applications are built. There are too many details that can break and each change needs to be tested on different browsers to do it all manually.

Personally, i believe there will be both models in the future. The first approach (JS enritched server controlled applications) will probably stay around for CMS sites with little user interaction.
But the more interesting model for me is the second one. This model opens a whole new world of feature ideas but also technical design and architectural challenges for the next years.

A few minutes ago, i accidentially found an old "Tamagotchi" i got as a present quite some years ago. When i thought about it, i saw an analogon to my current use of social networks: The Tamagotchi needs attention. It requires you to think about it all the time. You just needed to invest a couple of seconds every now and then, and if you did it right, you saw it grow and prosper. Now isnt that the same principle with you Facebook account? You log in 5 times a day, look what your friends are doing, give it some food represented by tweets or picture-uploads and then leave it alone. You see your network grow and prosper over time and just as a Tamagotchi, you build something up by investing just a minute here and one there. Does that thought lead to something? Deeper understanding? No. Probably not. But i  gave my Blog-Tamagotchi a little food again...

A couple of years ago, everybody was fascinated by automating build processes with ANT. This more modern version of MAKE worked with XML (how hipp) and Java. Instead of the old shell-level make environment, everything was contained in nice java packages and quite extendible. Some PHP users (including me) hopped on that train to automate packaging and some build-like PHP postprocessing. With PHING, even a PHP version of ANT is available. Nowadays the java guys have moved forward. Martin Fowler described in an article in 05 the benefits of the Ruby driven domain specific scripting language called RAKE over ANT and MAKE. In other places you read that build scripting has large benefits over configuration based systems especially for large systems. It seems like innovation is more and more driven by Ruby these days...

Since i am a PHP guy, i asked myself a few questions instead of just repeating the Java world (which i used to be part of some years ago, but now things are different).

What do we need from a build system in PHP?

- SVN support

- ZIP support

- Embedding of a JS compressor (ok, that was not easy, needed rhino which is java again)

- manipulating / generating config files out of configuration databases

- logging of deployment processes

- building RPMs

- transferring packages to remote systems

- calling web services to unpack there

So guess what: I think the perfect scripting language for this job is: PHP!

We built a small management system and some scripts in PHP and voilà: It works fine. Even more interesting: No special knowhow needed. Everyone can look in the code and understands how the build process works. And virtually no limits! We even fake a Browser and use the customer's web based upload service directly from our build system. And if we need to change the process: It's easy, all PHP.

I think we have a good solution now...

It turned out that the Q8200 processor i had in my machine at home was the only 4-core from intel without virtualization support. Therefore i could not run 64Bit clients in VMWare or Vitrualbox.

So i went into a shop and bought a shiny, new Q9400 instead. I went home, installed it and started my system, expecting more power and 64-Bit Support. But the effect was that neither VMWare 2.0 nor Virtualbox could start an image anymore. Not even 32 Bit images. A little research showed what the problem was: The kvm kernel module that comes with Ubuntu by default, could not be loaded with the old processor because of missing hardware support. With the new CPU, it loaded. Obviously the kernel module prevents the other virtualization engines to work.

De-Installing KVM was no option for me because i wanted to use it also for some tests. Therefore i searched for other options and found one in the ubuntu forum.

The solution that worked for me was to just unload the kernel modules before starting VirtualBox:

modprobe -r kvm_intel
modprobe -r kvm

Gestern abend zappe ich durchs Fernsehen und was sehe ich? Nachdem in Hart aber Fair eine Diskussion über das freiwillige "Entbößen" in den Medien zu einer Internet-Verständnisdiskussion abglitt, kommt auf Phönix mein alter Bekannter Alvar Freude in Diskussion mit Ossi Urchs und einer als Internetkritikerin auftretenden Journalistin zum Thema "Außer Kontrolle. Welche Macht hat das Internet". "Werden „Twitter“ und „Facebook“ reale menschliche Beziehungen immer mehr ersetzen? Und wie viel Freiheit verträgt das Internet überhaupt?" Da gibt es wohl einiges gesellschaftlich aufzuarbeiten...

Politiker gehören offenbar - trotz ihrer Kommunikationslastigen Arbeit - zu den eMail-Ausdruckern in der Internet-Bedürfnispyramide. Aber auch das scheint sich langsam zu ändern. Während hierzulande offenbar Ignoranz und Unverständnis vorherrschen, gewinnt man in den USA Wahlen durch konsequenten Einsatz von Blogs, Twitter, Video-Plattformen...

Erst gestern hatte ich in unserer Firmenküche mit einem Kollegen eine hitzige Diskussion über Urheberrecht im Internetzeitalter. Und heute macht mich Lars Jankowfsky auf einen genau passenden Blog-Eintrag von Kristian Köhntopp aufmerksam, der auf den Punkt bringt, warum die Diskussion so zäh verlief: Meine Sichtweise über den "Impact" des Internets und die offensichtlich früher oder später eintretenden Folgen für Wirtschaft und Gesellschaft entspringt einem Blickwinkel, den einfach nicht alle teilen.

Wobei ich nicht glaube, dass es eine reine Frage von biologischem Alter ist. Für mich ist es eher eine Frage von Paralleluniversen. Es gibt Menschen, die solange sie denken, digitale Medien nutzen (ich habe dazu auch mal den zeitgeistigen Begriff "digital Native" gehört) aber trotzdem nie da hin kommen werden, wo Kristian heute ist.

Warum? Weil sie nicht _ganz_ drin sind. Kristian arbeitet im/am Netz, kommuniziert mit lauter Leuten die das auch tun, rund um die Welt. Alle mit einer - angenommen - ausgeprägten Abstraktionsgabe. Werden in Zukunft alle Menschen so sein?

Es wird immer noch Leute geben, deren Broterwerb außerhalb des allumfassenden Kommunikationsmediums stattfindet. Und welche, die sich da einfach unwohl fühlen, so wie es Leute gibt, die sich mit dem Auto nicht in die Großstadt trauen. Wo ich aber absolut zustimme ist, dass in Bereichen wie Copyright und Persönlichkeitsrechten (Recht am eigenen Bild etc.) noch einige kontroverse Diskussionen mit Protagonisten aus den unterschiedlichen "Lagern" zu erwarten sind.

After the air guitar contests have become mainstream, the never sleeping trend scouts have spotted a new upcoming hype: Air Sex. What this means? Official contest website here. In the news: here. I wonder what the contestants do before and after the show...

never thought that we had anything in common. But look at this (subtitles)! 

I just found a funny post at Jeff Atwood's blog about error handling.

With Exceptions being my favourite add-on in PHP 5, i loved this post.

Error handling is something that often is regarded as something that is somehow needed but a simple bailout often appears to do it. The more complex your application gets, the more important it becomes to do the right error handling and logging.

This goes into both directions: excessive error logging results in spending too much money on hard disks and hides the real problems between gigabytes of "user uploaded file with name collsion" messages. Therefore it is important to define a robust, consistent error handling and logging ruleset at the beginning of the development cycle. Adding error handling while coding is quite simple, doing it afterwards is a nightmare. And debugging an application with non-adequate error handling and logging is a real pain.

In my world, errors are in general reported with exceptions, of course. But only if they are of interest for the surrounding application of course, not if they are handled locally. So i wrap all system calls and php calls that return "false" in case of error in classes and let them evaluate the error response and throw an exception.

For handling these exceptions, we have some rules where i come from:

1. Let exceptions fall. Dont catch them if you cant handle them. Never ever catch an exception and throw another one just to cast them. You'll loose the stack trace and any additional information. (in PHP 5.3 we will hopefully be able to cast an exception by throwing another one that is constructed from the original exception)

2. Define a common Exception Log method that produces a standard output of your exception including stack trace and extra data.

3. Dont log errors when the Exception is thrown, but log them on the top-level if nobody has catched them. This way you prevent log output for catched exception.

4. Give exceptions a code. Define a catalogue of all exception codes in a common place. This way you can handle a group of exceptions that same way but still know the individual differences

5. Define a last defense line where all exceptions are caught so under no circumstances a user ever gets to see one. Instead, catch all exceptions there and output something like "Come back later". For responses to Ajax calls, transcode the exceptions to a JSON repsonse (see 6.)

6. In a Javascript headed application, make sure to have a transparent transport of error messages from server to client (in a JSON structure that is present in each result of Ajax calls or via comet or the like)

7. Throw Server-Generated error messages and Javascript generated error messages into an openAjax-Hub and make some Widget listen to this hub to display the actual messages.

And then: Excpetion driven development comes in. Whenever you have an error that was a hell to find, make sure to improve the error handling around it so it becomes as you would have needed if before. Do this before fixing the actual bug, so you can test the error output.

I did this a lot in our current 270,000 LOC application and it made it more easy to debug every time.